nx_q5_0_fused_dot_gate.nx
buildroot/runtime/nx_q5_0_fused_dot_gate.nx
about
nx_q5_0_fused_dot_gate.nx -- ISOLATED proof of the model-appropriate
SOTA decode lever: a FUSED Q5_0 dequant-dot (read the 22-byte block,
dequant in-register, dot) vs the CURRENT path (dequant the whole row
to F32 i64-slots, then dot).
WHY: nx_gguf_typecensus shows our model is 79% Q5_0, but the loader
dequants it to F32 i64-slots (8 B/value) -- the memory-bound decode
matmul then reads 8 B/value when Q5_0 is 0.69 B/value native = ~11.6x
more memory than needed. SOTA (Marlin) keeps weights quantized and
dequants in-register. This gate proves the fused kernel bit-exact +
measures the memory-bound win at the real FFN shape. Ratio within one
run => host-noise-immune. NO forward changes (no " Paris" risk).
Checks:
1 fused q5_0 dot == dequant-to-f32-then-dot, bit-exact (whole matmul)
2 fused reads ~11.6x fewer bytes; measure the speedup (floor 1.5x)
lineage_id: q5_0_fused_dot_gate_v1
dependencies 7 imports · 0 importers
imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_f32.nxnx_f32_cvt.nxnx_q5_0_to_f32.nxnx_fmt.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 28 | const QK: i64 = 896 // hidden (reduction dim), div by 32 |
| 29 | const QN: i64 = 4864 // ffn (W_gate n) |
| 30 | const QREPS: i64 = 60 |
| 31 | const QFLOOR_X100: i64 = 150 |
| 34 | const Q5B: i64 = 22 |
| 35 | const Q5V: i64 = 32 |
functions
| 37 | func q_lcg(s: i64) -> i64 |
| 45 | func q_fill_weight(w: *u8, seed: i64) -> i64 |
| 66 | func q_fill_a(p: *i64, count: i64, seed: i64) -> i64 |
| 81 | func q5_0_dot(qbuf: *u8, qoff: i64, A: *i64, k: i64, vscr: *i64) -> i64 called by 1: main calls 7: nx_f16_to_f32nx_le_read_u16nx_le_read_u32nx_le_read_u8nx_f32_mulnx_i32_to_f32+1 |
| 109 | func q_nl() -> i64 { fmt_puts("\n" as *u8); return 0 } |
| 111 | func main() -> i64 |