nx_q4k_fused_vs_x4_gate.nx
buildroot/runtime/nx_q4k_fused_vs_x4_gate.nx
about
nx_q4k_fused_vs_x4_gate.nx -- THE HEAD-TO-HEAD the SOTA decision actually rests on:
the FUSED INTEGER dequant-dot (nx_q4k_dot_row_col) against the LIVE forward kernel
(fq4m_rows_x4 / nx_f32_q4k_matmul_x4), in ONE binary, ONE shape, ONE thread.
WHY THIS GATE EXISTS. nx_q4k_speed_bench reports the fused integer route beating f32 by ~20-48x, but
its f32 leg is SINGLE-THREADED SCALAR EMULATED f32 -- which is NOT the live path. The live path is
nx_f32_q4k_matmul_pool_x4: hardware __f32x4_dot AND multicore. Quoting that 20-48x as the shipped win
would repeat the exact "10.25x vs SERIAL scalar" error already caught once in this lane. A speedup is
only real against WHAT IS ACTUALLY DISPATCHED.
THREADING IS DELIBERATELY EXCLUDED, AND THAT IS SOUND. Both routes band-parallelise over the SAME axis
(output columns j) with identical banding, so a pool multiplies both sides equally and the
single-thread ratio carries over. It is also CONSERVATIVE for the fused side: fused reads only the
packed Q4_K bytes while x4 materialises k f32 values per column, so under real multicore memory
pressure fused should scale strictly BETTER, never worse. Any win measured here is a LOWER BOUND.
NUMERIC REGIME (why bit-exact comparison is legitimate here, and why that is NOT a general claim).
Weights are synthetic Q4_K super-blocks with d=1.0, dmin=0, all scales=1, mins=0, so every dequantised
weight is an exact integer 0..15. The activation column holds small exact integers. Then:
x4 computes an exact-integer f32 sum (|dot| <= 15*3*1024 = 46080 << 2^24, so f32 add is EXACT).
fused computes d1=2^24, m1=0, so v = 2^24*q4 (Q24); v*col_q10 is Q34; >>24 recovers Q10 exactly.
Both therefore represent the SAME integer and must agree BIT-EXACTLY. On REAL model weights they will
NOT -- fused is activation quantisation (W4A-fixed), a genuine precision change. This gate proves the
kernels compute the same function; it does NOT license dispatch on real weights without a separate
fidelity tooth (nx_q4k_ggml_kat's 2% band is the existing instrument for that).
genealogy_id: nx_q4k_x8_gate (data construction, exact regime, refutation-tooth discipline)
dependencies 6 imports · 0 importers
imports: nx_f32_q4k_matmul.nxnx_q4k_matmul.nxnx_dequant_iter.nxnx_f32_cvt.nxnx_fmt.nxnx_gate_verdict.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 35 | const FVX_M: i64 = 1 // decode shape -- the dominant serving case and where fused should be strongest |
| 36 | const FVX_K: i64 = 1024 |
| 37 | const FVX_N: i64 = 512 |
functions
| 39 | func fv_puts(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 } |
| 40 | func fv_num(v: i64) -> i64 |
| 50 | func fv_lcg(s: i64) -> i64 { var v: i64 = s * 1103515245 + 12345; v = v & 2147483647; return v } |
| 53 | func fv_block(buf: *u8, off: i64, seed: i64) -> i64 |
| 70 | func fv_fill_w(buf: *u8, n_rows: i64, k: i64, seed: i64) -> i64 |
| 82 | func main(argc: i64, argv: *i64) -> i64 |