code wiki / (root) / nx_dequant_iter.nx

nx_dequant_iter.nx

buildroot/runtime/nx_dequant_iter.nx

11194 B297 linesdepth 6pulls 8 transitivereach 85 importersview sourcekind librarytopic dequant
docsdependenciesstructsconstsfunctions

about

nx_dequant_iter.nx -- per-block Q4_K dequant iterator. Architectural step toward conductor Phase C (dispatcher with fused dequant+matmul). Instead of materializing a whole tensor's worth of Q10 i64 storage, the dispatcher will hold a small per-block iterator that amortizes the 8-scale-8-min unpack across the 256 value reads from that block. A matmul kernel asks the iterator for individual values (or sub-block batches) on demand. Why this matters (HONESTLY, no perf claim attached): The existing nx_gguf_dequant_q4_k writes ALL 256 values of every super-block into a flat output buffer. For Llama-7B's ~7B params at 256 values/block that's ~27 million blocks materialized eagerly. At ~11 ns/value (qemu-riscv64 measurement from nx_dequant_bench, 2026-05-19) that's ~77 seconds of pure dequant if the substrate ever materializes the full model. The iterator pattern (this brick) DOES NOT make per-value dequant faster. It enables the SCAFFOLDING for a future dispatcher that dequants only the blocks a given matmul tile reads. Whether the dispatcher delivers net latency wins vs the materialize-everything path depends on the matmul access pattern and remains to be measured -- not claimed here. What this brick DOES claim, narrowly: - One-time per-block setup: read f16 d + dmin, unpack 8 scales + 8 mins from the 12-byte packed header (one constant-time pass) - Constant-time per-value access: nx_q4k_iter_value(it, idx) does index math + nibble extract + (d * sc) * q - dmin * m - Bit-exact match against nx_gguf_dequant_q4_k for the same block bytes (verified by smoke) genealogy_id: ggml_q4k_block_canon + iterator_pattern_canonical lineage_id: substrate_dequant_iter_v1_q4k

dependencies 5 imports · 17 importers

nx_syscalls.nx nx_tier.nx nx_le.nx nx_gguf.nx nx_gguf_load.nx nx_dequant_iter.nx nx_dequant_iter_bench_test.nx nx_dequant_iter_test.nx nx_f32_q4k_matmul.nx nx_q4k_dot_simd.nx nx_q4k_fused_vs_x4_gate.nx nx_q4k_gemm_mt.nx nx_q4k_ggml_kat.nx nx_q4k_linear.nx nx_q4k_linear_hp.nx nx_q4k_matmul.nx

diagram shows first 10 each side; +0 more imports, +7 more importers in the complete lists below.

imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_gguf.nxnx_gguf_load.nx

imported by: nx_dequant_iter_bench_test.nxnx_dequant_iter_test.nxnx_f32_q4k_matmul.nxnx_q4k_dot_simd.nxnx_q4k_fused_vs_x4_gate.nxnx_q4k_gemm_mt.nxnx_q4k_ggml_kat.nxnx_q4k_linear.nxnx_q4k_linear_hp.nxnx_q4k_matmul.nxnx_q4k_matmul_bench_test.nxnx_q4k_matmul_test.nxnx_q4k_real_gemm.nxnx_q4k_speed_bench.nxnx_qwen_hybrid_attn.nxnx_qwen_hybrid_ffn.nxnx_qwen_hybrid_qkv.nx

structs

48struct NxQ4KBlockIter

consts

70const NX_Q4K_ITER_BYTES: nx_int = 152 // 19 * 8
72const NX_Q4KI_OK: nx_int = 0
73const NX_Q4KI_ERR_BAD_BUF: nx_int = 1
74const NX_Q4KI_ERR_BAD_IDX: nx_int = 2
75const NX_Q4KI_N_VERDICTS: nx_int = 3

functions

77func nx_q4ki_verdict_is_valid(v: nx_int) -> nx_int
called by 1: main
85func nx_q4k_iter_alloc() -> *NxQ4KBlockIter
104func nx_q4k_iter_init(buf: *u8, super_off: i64, it: *NxQ4KBlockIter) -> nx_int
158func _q4k_iter_scale_at(it: *NxQ4KBlockIter, sb: nx_int) -> i64
170func _q4k_iter_min_at(it: *NxQ4KBlockIter, sb: nx_int) -> i64
188func nx_q4k_iter_scale_min(it: *NxQ4KBlockIter, sb: nx_int,
209func nx_q4k_iter_scale(it: *NxQ4KBlockIter, sb: nx_int) -> i64
215func nx_q4k_iter_min(it: *NxQ4KBlockIter, sb: nx_int) -> i64
232func nx_q4k_iter_value(it: *NxQ4KBlockIter, idx: nx_int) -> i64
270func nx_q4k_iter_subblock_dequant(it: *NxQ4KBlockIter, sb: nx_int,