code wiki / (root) / nx_llm_forward_profile.nx

nx_llm_forward_profile.nx

buildroot/runtime/nx_llm_forward_profile.nx

5070 B105 linesdepth 13pulls 55 transitivereach 0 importersview sourcekind tooltopic llm
docsdependenciesstructsconstsfunctions

about

nx_llm_forward_profile.nx -- clean per-decode-forward timing on the REAL model, to find where a decode token's time actually goes. The 34-token bench is too noisy on this host (WSL bounces + multi- session load) to attribute sub-20% changes. This times ONE forward_v4 decode call at a time (no sampler/detok/penalty/loop overhead), after a warmup prefill so the dequant cache + lm_head pool are hot. Combined with the isolated lm_head number (nx_matmul_t_pool_gate: ~143ms pooled at m=1,n=151936), it splits the decode forward into lm_head vs the 24-layer block stack. Prints each step's us (watch for growth = KV-cache-length effect) and the average. No vocab/bpe load (forward needs neither). expect_exit: 0

dependencies 16 imports · 0 importers

nx_syscalls.nx nx_itoa_lib.nx nx_tier.nx nx_gguf.nx nx_gguf_load.nx nx_gguf_meta.nx nx_f32.nx nx_f32_kv_cache.nx nx_f32_lazy_weight.nx nx_f32_llama_block.nx nx_llm_forward_profile.nx

diagram shows first 10 each side; +6 more imports, +0 more importers in the complete lists below.

imports: nx_syscalls.nxnx_itoa_lib.nxnx_tier.nxnx_gguf.nxnx_gguf_load.nxnx_gguf_meta.nxnx_f32.nxnx_f32_kv_cache.nxnx_f32_lazy_weight.nxnx_f32_llama_block.nxnx_f32_llama_block_v4.nxnx_f32_llama_stack_v4.nxnx_f32_llama_layer_lazy_load.nxnx_f32_llm.nxnx_f32_llm_v4.nxnx_f32_llm_read_dims.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap sys_read_file sys_openat_rd sys_lseek sys_mmap ↻ sys_read sys_close pf_puts sys_write nx_gguf_parse nx_le_read_u32 nx_le_read_u64 nx_le_read_u32 ↻ _gguf_skip_value nx_le_read_u64 ↻ nx_le_read_u32 ↻ _gguf_skip_value ↻ sys_mmap ↻ nx_f32_llama_model_alloc sys_mmap ↻ nx_f32_llm_read_dims_from_ sys_mmap ↻ nx_gguf_meta_find nx_le_read_u64 ↻ nx_le_read_u32 ↻ _gmeta_bytes_equal nx_gguf_skip_value _gguf_skip_value ↻ nx_gguf_meta_read_string_l nx_le_read_u64 ↻ nx_gguf_meta_read_string_p _fld_read_arch_u32 sys_mmap ↻ _fld_concat_key nx_gguf_meta_find ↻ nx_gguf_meta_read_u32 nx_le_read_u32 ↻ _fld_read_vocab_size_from_ sys_mmap ↻

structs

none

consts

none

functions

34func pf_puts(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(1,s,n); return 0 }
called by 2: pf_nlmain calls 1: sys_write
39func pf_num(v: i64) -> i64 { nxi_out(v); return 0 }
called by 1: main calls 1: nxi_out
40func pf_nl() -> i64 { pf_puts("\n" as *u8); return 0 }
called by 1: main calls 1: pf_puts
42func main(argc: i64, argv: *i64) -> i64