nx_dequant_iter.nx
buildroot/runtime/nx_dequant_iter.nx
about
nx_dequant_iter.nx -- per-block Q4_K dequant iterator.
Architectural step toward conductor Phase C (dispatcher with fused
dequant+matmul). Instead of materializing a whole tensor's worth
of Q10 i64 storage, the dispatcher will hold a small per-block
iterator that amortizes the 8-scale-8-min unpack across the 256
value reads from that block. A matmul kernel asks the iterator
for individual values (or sub-block batches) on demand.
Why this matters (HONESTLY, no perf claim attached):
The existing nx_gguf_dequant_q4_k writes ALL 256 values of every
super-block into a flat output buffer. For Llama-7B's ~7B params
at 256 values/block that's ~27 million blocks materialized eagerly.
At ~11 ns/value (qemu-riscv64 measurement from nx_dequant_bench,
2026-05-19) that's ~77 seconds of pure dequant if the substrate
ever materializes the full model.
The iterator pattern (this brick) DOES NOT make per-value dequant
faster. It enables the SCAFFOLDING for a future dispatcher that
dequants only the blocks a given matmul tile reads. Whether the
dispatcher delivers net latency wins vs the materialize-everything
path depends on the matmul access pattern and remains to be
measured -- not claimed here.
What this brick DOES claim, narrowly:
- One-time per-block setup: read f16 d + dmin, unpack 8 scales +
8 mins from the 12-byte packed header (one constant-time pass)
- Constant-time per-value access: nx_q4k_iter_value(it, idx)
does index math + nibble extract + (d * sc) * q - dmin * m
- Bit-exact match against nx_gguf_dequant_q4_k for the same
block bytes (verified by smoke)
genealogy_id: ggml_q4k_block_canon + iterator_pattern_canonical
lineage_id: substrate_dequant_iter_v1_q4k
dependencies 5 imports · 17 importers
diagram shows first 10 each side; +0 more imports, +7 more importers in the complete lists below.
imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_gguf.nxnx_gguf_load.nx
imported by: nx_dequant_iter_bench_test.nxnx_dequant_iter_test.nxnx_f32_q4k_matmul.nxnx_q4k_dot_simd.nxnx_q4k_fused_vs_x4_gate.nxnx_q4k_gemm_mt.nxnx_q4k_ggml_kat.nxnx_q4k_linear.nxnx_q4k_linear_hp.nxnx_q4k_matmul.nxnx_q4k_matmul_bench_test.nxnx_q4k_matmul_test.nxnx_q4k_real_gemm.nxnx_q4k_speed_bench.nxnx_qwen_hybrid_attn.nxnx_qwen_hybrid_ffn.nxnx_qwen_hybrid_qkv.nx
structs
| 48 | struct NxQ4KBlockIter |
consts
| 70 | const NX_Q4K_ITER_BYTES: nx_int = 152 // 19 * 8 |
| 72 | const NX_Q4KI_OK: nx_int = 0 |
| 73 | const NX_Q4KI_ERR_BAD_BUF: nx_int = 1 |
| 74 | const NX_Q4KI_ERR_BAD_IDX: nx_int = 2 |
| 75 | const NX_Q4KI_N_VERDICTS: nx_int = 3 |
functions
| 77 | func nx_q4ki_verdict_is_valid(v: nx_int) -> nx_int called by 1: main |
| 85 | func nx_q4k_iter_alloc() -> *NxQ4KBlockIter |
| 104 | func nx_q4k_iter_init(buf: *u8, super_off: i64, it: *NxQ4KBlockIter) -> nx_int |
| 158 | func _q4k_iter_scale_at(it: *NxQ4KBlockIter, sb: nx_int) -> i64 |
| 170 | func _q4k_iter_min_at(it: *NxQ4KBlockIter, sb: nx_int) -> i64 |
| 188 | func nx_q4k_iter_scale_min(it: *NxQ4KBlockIter, sb: nx_int, |
| 209 | func nx_q4k_iter_scale(it: *NxQ4KBlockIter, sb: nx_int) -> i64 called by 4: fq4m_rows_fusedfq4m_rows_i8simdfq4m_fill_q24nx_q4k_dot_row_col calls 1: _q4k_iter_scale_at |
| 215 | func nx_q4k_iter_min(it: *NxQ4KBlockIter, sb: nx_int) -> i64 called by 4: fq4m_rows_fusedfq4m_rows_i8simdfq4m_fill_q24nx_q4k_dot_row_col calls 1: _q4k_iter_min_at |
| 232 | func nx_q4k_iter_value(it: *NxQ4KBlockIter, idx: nx_int) -> i64 |
| 270 | func nx_q4k_iter_subblock_dequant(it: *NxQ4KBlockIter, sb: nx_int, |