code wiki / (root) / nx_f32_llm.nx

nx_f32_llm.nx

buildroot/runtime/nx_f32_llm.nx

5686 B158 linesdepth 10pulls 19 transitivereach 55 importersview sourcekind librarytopic f32
docsdependenciesstructsconstsfunctions

about

nx_f32_llm.nx -- full Llama-class model forward pass. The API a real text-in -> logits-out inference loop calls. Composes: embed lookup (token_id -> embedding row copy) nx_f32_llama_stack (N-layer transformer forward) nx_f32_rmsnorm (final pre-LM-head normalization) nx_f32_matmul (LM head: hidden -> logits) Algorithm: 1. Embedding lookup: for each token t in 0..n_tokens: x[t, :] = embed_weights[token_ids[t], :] 2. Stack forward through N layers (advances cache by n_tokens). 3. Final RMSNorm per token with gamma_out. 4. Logits = matmul(out_norm, lm_head) -> [n_tokens, vocab_size] The caller reads logits[(n_tokens-1)*vocab_size:n_tokens*vocab_size] to get next-token logits for autoregressive sampling. Earlier rows are valid for any per-position decode/classification. Tied-embed optimization: real Qwen/Llama variants often use the same matrix for embedding and LM head. Caller can pass the transposed embed_weights as lm_head -- this kernel does not care. genealogy_id: standard_llama_forward_pass lineage_id: substrate_f32_llm_v1

dependencies 8 imports · 51 importers

nx_syscalls.nx nx_tier.nx nx_f32.nx nx_f32_rmsnorm.nx nx_f32_matmul.nx nx_f32_kv_cache.nx nx_f32_llama_block.nx nx_f32_llama_stack.nx nx_f32_llm.nx nx_batched_gate.nx nx_batched_tput.nx nx_chat_warmcache.nx nx_coder_gen_f32.nx nx_companion_chat.nx nx_companion_voice.nx nx_f32_llama_v4b.nx nx_f32_llama_v4p.nx nx_f32_llm_bench_test.nx nx_f32_llm_chat_test.nx

diagram shows first 10 each side; +0 more imports, +41 more importers in the complete lists below.

imports: nx_syscalls.nxnx_tier.nxnx_f32.nxnx_f32_rmsnorm.nxnx_f32_matmul.nxnx_f32_kv_cache.nxnx_f32_llama_block.nxnx_f32_llama_stack.nx

imported by: nx_batched_gate.nxnx_batched_tput.nxnx_chat_warmcache.nxnx_coder_gen_f32.nxnx_companion_chat.nxnx_companion_voice.nxnx_f32_llama_v4b.nxnx_f32_llama_v4p.nxnx_f32_llm_bench_test.nxnx_f32_llm_chat_test.nxnx_f32_llm_cohere_test.nxnx_f32_llm_e2e_test.nxnx_f32_llm_embed_diag_test.nxnx_f32_llm_live_dims_test.nxnx_f32_llm_live_gen_test.nxnx_f32_llm_live_load_only_test.nxnx_f32_llm_live_load_test.nxnx_f32_llm_load.nxnx_f32_llm_load_test.nxnx_f32_llm_read_dims.nxnx_f32_llm_read_dims_test.nxnx_f32_llm_run.nxnx_f32_llm_run_test.nxnx_f32_llm_run_v2.nxnx_f32_llm_run_v2_test.nxnx_f32_llm_run_v3.nxnx_f32_llm_run_v3_test.nxnx_f32_llm_serve.nxnx_f32_llm_test.nxnx_f32_llm_v4.nxnx_f32_llm_v4_test.nxnx_f32_qwen_capital_gate.nxnx_llm_argmax_probe.nxnx_llm_batch_serve.nxnx_llm_diff_probe.nxnx_llm_forward_profile.nxnx_llm_france_probe.nxnx_llm_layerlens_probe.nxnx_llm_sched.nxnx_llm_sched_gate.nxnx_lowrank_kv_real.nxnx_paged_fwd_gate.nxnx_qwen_native_gate.nxnx_qwen_wsl_timing_gate.nxnx_reason_paged_probe.nxnx_reason_probe.nxnx_reasoning.nxnx_reasoning_gate.nxnx_reasoning_paged.nxnx_specdec.nxnx_specdec_gate.nx

structs

51struct NxF32LlamaModel

consts

38const NX_F32_LLM_OK: nx_int = 0
39const NX_F32_LLM_ERR_BAD_DIM: nx_int = 1
40const NX_F32_LLM_ERR_NULL: nx_int = 2
41const NX_F32_LLM_ERR_TOKEN: nx_int = 3
42const NX_F32_LLM_ERR_STACK: nx_int = 4
43const NX_F32_LLM_N_VERDICTS: nx_int = 5
66const NX_F32_LLAMA_MODEL_BYTES: nx_int = 96 // 12 fields * 8

functions

45func nx_f32_llm_verdict_is_valid(v: nx_int) -> nx_int
called by 1: main
68func nx_f32_llama_model_alloc() -> *NxF32LlamaModel
called by 42: mainmainmainmainmainmain+36 calls 1: sys_mmap
77func nx_f32_llm_forward(
141func nx_f32_llm_argmax_last(logits: *i64, n_tokens: nx_int,
called by 1: main calls 1: nx_f32_lt