code wiki / (root) / nx_f32_llama_block_v4.nx

nx_f32_llama_block_v4.nx

buildroot/runtime/nx_f32_llama_block_v4.nx

14493 B389 linesdepth 10pulls 40 transitivereach 49 importersview sourcekind librarytopic f32
docsdependenciesstructsconstsfunctions

about

nx_f32_llama_block_v4.nx -- per-layer forward with lazy weights. Mirrors nx_f32_llama_block (v3) but accepts NxF32LazyWeight for each of the 7 matmul weights, dispatching to f32 or Q4_K backends per tensor. RMSNorm gammas remain as raw *i64 (they're 1D vectors, always small enough that lazy dequant isn't needed). When real GGUF files have most weights as Q4_K (e.g. Qwen2.5-0.5B- Instruct Q4_K_M), this forward avoids the eager-dequant memory blow-up: each matmul streams the Q4_K bytes through nx_q4k_to_f32 one row at a time, peak working memory drops from O(k*n) to O(n). genealogy_id: vaswani_2017 + touvron_2023_llama + ggml_format_taxonomy + tagged_dispatch lineage_id: substrate_f32_llama_block_v4_lazy

dependencies 13 imports · 42 importers

nx_syscalls.nx nx_tier.nx nx_f32.nx nx_f32_rmsnorm.nx nx_f32_matmul.nx nx_f32_activations.nx nx_f32_rope.nx nx_f32_attn_multi.nx nx_f32_kv_cache.nx nx_f32_attn_cached.nx nx_f32_llama_block_v4.nx nx_batched_gate.nx nx_batched_tput.nx nx_chat_warmcache.nx nx_coder_gen_f32.nx nx_companion_chat.nx nx_companion_voice.nx nx_f32_llama_block_v4_test.nx nx_f32_llama_layer_lazy_load.nx nx_f32_llama_layer_lazy_load_test. nx_f32_llama_stack_v4.nx

diagram shows first 10 each side; +3 more imports, +32 more importers in the complete lists below.

imports: nx_syscalls.nxnx_tier.nxnx_f32.nxnx_f32_rmsnorm.nxnx_f32_matmul.nxnx_f32_activations.nxnx_f32_rope.nxnx_f32_attn_multi.nxnx_f32_kv_cache.nxnx_f32_attn_cached.nxnx_f32_lazy_weight.nxnx_thread_pool.nxnx_f32_llama_block.nx

imported by: nx_batched_gate.nxnx_batched_tput.nxnx_chat_warmcache.nxnx_coder_gen_f32.nxnx_companion_chat.nxnx_companion_voice.nxnx_f32_llama_block_v4_test.nxnx_f32_llama_layer_lazy_load.nxnx_f32_llama_layer_lazy_load_test.nxnx_f32_llama_stack_v4.nxnx_f32_llama_stack_v4_test.nxnx_f32_llama_v4b.nxnx_f32_llama_v4p.nxnx_f32_llm_bench_test.nxnx_f32_llm_chat_test.nxnx_f32_llm_cohere_test.nxnx_f32_llm_embed_diag_test.nxnx_f32_llm_live_gen_test.nxnx_f32_llm_live_load_only_test.nxnx_f32_llm_live_load_test.nxnx_f32_llm_probe.nxnx_f32_llm_run_v3.nxnx_f32_llm_run_v3_test.nxnx_f32_llm_serve.nxnx_f32_llm_v4.nxnx_f32_llm_v4_test.nxnx_f32_qwen_capital_gate.nxnx_llm_argmax_probe.nxnx_llm_batch_serve.nxnx_llm_diff_probe.nxnx_llm_forward_profile.nxnx_llm_france_probe.nxnx_llm_layerlens_probe.nxnx_llm_sched_gate.nxnx_lowrank_kv_real.nxnx_paged_fwd_gate.nxnx_qwen_native_gate.nxnx_qwen_wsl_timing_gate.nxnx_reason_paged_probe.nxnx_reason_probe.nxnx_reasoning_gate.nxnx_specdec_gate.nx

structs

44struct NxF32LlamaLayerLazy
117struct NxActCtx

consts

30const NX_MAGIC_2026: i64 = 2026
32const NX_BLK4_OK: nx_int = 0
33const NX_BLK4_ERR_BAD_DIM: nx_int = 1
34const NX_BLK4_ERR_NULL: nx_int = 2
35const NX_BLK4_ERR_CACHE: nx_int = 3
36const NX_BLK4_N_VERDICTS: nx_int = 4
59const NX_F32_LLAMA_LAYER_LAZY_BYTES: nx_int = 96 // 12 fields * 8
70const BP_ALLOC: i64 = 0
71const BP_RMSNORM: i64 = 1
72const BP_MATMUL: i64 = 2
73const BP_ROPE: i64 = 3
74const BP_ATTN: i64 = 4
75const BP_ACT: i64 = 5
76const BP_RESID: i64 = 6
127const NX_ACT_CTX_BYTES: i64 = 64

functions

38func nx_blk4_verdict_is_valid(v: nx_int) -> nx_int
called by 1: main
61func nx_f32_llama_layer_lazy_alloc() -> *NxF32LlamaLayerLazy
81func nx_blk4_prof_enable(on: i64) -> i64
called by 1: main calls 1: sys_mmap
86func nx_blk4_prof_reset() -> i64
called by 1: main
94func nx_blk4_prof_read(b: i64) -> i64
called by 1: main
99func bp_t() -> i64
103func bp_add(b: i64, t0: i64) -> i64
130func _swiglu_task(ctx_i: i64) -> i64
143func nx_blk4_swiglu_pool(gate: *i64, up: *i64, out: *i64, count: i64) -> i64
172func _rope_task(ctx_i: i64) -> i64
190func nx_blk4_rope_heads_pool(base: i64, n_heads: i64, head_dim: i64,
220func nx_f32_llama_block_forward_v4(