code wiki / (root) / nx_q5_0_fused_dot_gate.nx

nx_q5_0_fused_dot_gate.nx

buildroot/runtime/nx_q5_0_fused_dot_gate.nx

7118 B198 linesdepth 5pulls 8 transitivereach 0 importersview sourcekind gate/prooftopic q5
docsdependenciesstructsconstsfunctions

about

nx_q5_0_fused_dot_gate.nx -- ISOLATED proof of the model-appropriate SOTA decode lever: a FUSED Q5_0 dequant-dot (read the 22-byte block, dequant in-register, dot) vs the CURRENT path (dequant the whole row to F32 i64-slots, then dot). WHY: nx_gguf_typecensus shows our model is 79% Q5_0, but the loader dequants it to F32 i64-slots (8 B/value) -- the memory-bound decode matmul then reads 8 B/value when Q5_0 is 0.69 B/value native = ~11.6x more memory than needed. SOTA (Marlin) keeps weights quantized and dequants in-register. This gate proves the fused kernel bit-exact + measures the memory-bound win at the real FFN shape. Ratio within one run => host-noise-immune. NO forward changes (no " Paris" risk). Checks: 1 fused q5_0 dot == dequant-to-f32-then-dot, bit-exact (whole matmul) 2 fused reads ~11.6x fewer bytes; measure the speedup (floor 1.5x) lineage_id: q5_0_fused_dot_gate_v1

dependencies 7 imports · 0 importers

nx_syscalls.nx nx_tier.nx nx_le.nx nx_f32.nx nx_f32_cvt.nx nx_q5_0_to_f32.nx nx_fmt.nx nx_q5_0_fused_dot_gate.nx

imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_f32.nxnx_f32_cvt.nxnx_q5_0_to_f32.nxnx_fmt.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap q_fill_weight q_lcg q_fill_a q_lcg ↻ nx_i32_to_f32 nx_q5_0_to_f32 _nx_q5_0_block_to_f32 nx_le_read_u16 nx_f16_to_f32 nx_le_read_u32 nx_le_read_u8 nx_f32_add nx_f32_classify nx_f32_exp_field nx_f32_mant_field nx_f32_sign nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_f32_mul nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_mant_field ↻ nx_f32_exp_field ↻ q5_0_dot nx_f16_to_f32 ↻ nx_le_read_u16 ↻ nx_le_read_u32 ↻ nx_le_read_u8 ↻ nx_f32_mul ↻ nx_i32_to_f32 ↻ nx_f32_add ↻ fmt_puts fmt_puts_fd sys_write q_nl fmt_puts ↻ sys_now_us sys_mmap ↻

structs

none

consts

28const QK: i64 = 896 // hidden (reduction dim), div by 32
29const QN: i64 = 4864 // ffn (W_gate n)
30const QREPS: i64 = 60
31const QFLOOR_X100: i64 = 150
34const Q5B: i64 = 22
35const Q5V: i64 = 32

functions

37func q_lcg(s: i64) -> i64
45func q_fill_weight(w: *u8, seed: i64) -> i64
called by 1: main calls 1: q_lcg
66func q_fill_a(p: *i64, count: i64, seed: i64) -> i64
called by 1: main calls 2: q_lcgnx_i32_to_f32
81func q5_0_dot(qbuf: *u8, qoff: i64, A: *i64, k: i64, vscr: *i64) -> i64
109func q_nl() -> i64 { fmt_puts("\n" as *u8); return 0 }
called by 1: main calls 1: fmt_puts
111func main() -> i64