code wiki / (root) / nx_q4k_fused_vs_x4_gate.nx

nx_q4k_fused_vs_x4_gate.nx

buildroot/runtime/nx_q4k_fused_vs_x4_gate.nx

13318 B237 linesdepth 9pulls 24 transitivereach 0 importersview sourcekind gate/prooftopic q4k
docsdependenciesstructsconstsfunctions

about

nx_q4k_fused_vs_x4_gate.nx -- THE HEAD-TO-HEAD the SOTA decision actually rests on: the FUSED INTEGER dequant-dot (nx_q4k_dot_row_col) against the LIVE forward kernel (fq4m_rows_x4 / nx_f32_q4k_matmul_x4), in ONE binary, ONE shape, ONE thread. WHY THIS GATE EXISTS. nx_q4k_speed_bench reports the fused integer route beating f32 by ~20-48x, but its f32 leg is SINGLE-THREADED SCALAR EMULATED f32 -- which is NOT the live path. The live path is nx_f32_q4k_matmul_pool_x4: hardware __f32x4_dot AND multicore. Quoting that 20-48x as the shipped win would repeat the exact "10.25x vs SERIAL scalar" error already caught once in this lane. A speedup is only real against WHAT IS ACTUALLY DISPATCHED. THREADING IS DELIBERATELY EXCLUDED, AND THAT IS SOUND. Both routes band-parallelise over the SAME axis (output columns j) with identical banding, so a pool multiplies both sides equally and the single-thread ratio carries over. It is also CONSERVATIVE for the fused side: fused reads only the packed Q4_K bytes while x4 materialises k f32 values per column, so under real multicore memory pressure fused should scale strictly BETTER, never worse. Any win measured here is a LOWER BOUND. NUMERIC REGIME (why bit-exact comparison is legitimate here, and why that is NOT a general claim). Weights are synthetic Q4_K super-blocks with d=1.0, dmin=0, all scales=1, mins=0, so every dequantised weight is an exact integer 0..15. The activation column holds small exact integers. Then: x4 computes an exact-integer f32 sum (|dot| <= 15*3*1024 = 46080 << 2^24, so f32 add is EXACT). fused computes d1=2^24, m1=0, so v = 2^24*q4 (Q24); v*col_q10 is Q34; >>24 recovers Q10 exactly. Both therefore represent the SAME integer and must agree BIT-EXACTLY. On REAL model weights they will NOT -- fused is activation quantisation (W4A-fixed), a genuine precision change. This gate proves the kernels compute the same function; it does NOT license dispatch on real weights without a separate fidelity tooth (nx_q4k_ggml_kat's 2% band is the existing instrument for that). genealogy_id: nx_q4k_x8_gate (data construction, exact regime, refutation-tooth discipline)

dependencies 6 imports · 0 importers

nx_f32_q4k_matmul.nx nx_q4k_matmul.nx nx_dequant_iter.nx nx_f32_cvt.nx nx_fmt.nx nx_gate_verdict.nx nx_q4k_fused_vs_x4_gate.nx

imports: nx_f32_q4k_matmul.nxnx_q4k_matmul.nxnx_dequant_iter.nxnx_f32_cvt.nxnx_fmt.nxnx_gate_verdict.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main fv_puts fv_fill_w fv_lcg fv_block fv_lcg ↻ nx_i32_to_f32 nx_q4k_iter_alloc sys_mmap nx_f32_q4k_matmul_x4 sys_mmap ↻ _fq4m_pack_a fq4m_rows_x4 sys_mmap ↻ nx_q4k_to_f32_packed nx_q4k_block_to_f32_packed nx_le_read_u16 nx_f16_to_f32 nx_le_read_u8 nx_i32_to_f32 ↻ _q4kp_st4 sys_munmap sys_munmap ↻ nx_q4k_dot_row_col nx_q4k_iter_init nx_le_read_u16 ↻ _gguf_f16_to_q24 nx_le_read_u8 ↻ nx_q4k_iter_scale _q4k_iter_scale_at nx_q4k_iter_min _q4k_iter_min_at nx_le_read_u8 ↻ nx_q10_to_f32 nx_i32_to_f32 ↻ nx_q4km_q20_to_q10 fv_num fv_puts ↻ nx_f32_to_q10 sys_now_us

structs

none

consts

35const FVX_M: i64 = 1 // decode shape -- the dominant serving case and where fused should be strongest
36const FVX_K: i64 = 1024
37const FVX_N: i64 = 512

functions

39func fv_puts(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 }
called by 2: fv_nummain
40func fv_num(v: i64) -> i64
called by 1: main calls 1: fv_puts
50func fv_lcg(s: i64) -> i64 { var v: i64 = s * 1103515245 + 12345; v = v & 2147483647; return v }
called by 2: fv_blockfv_fill_w
53func fv_block(buf: *u8, off: i64, seed: i64) -> i64
called by 1: fv_fill_w calls 1: fv_lcg
70func fv_fill_w(buf: *u8, n_rows: i64, k: i64, seed: i64) -> i64
called by 1: main calls 2: fv_lcgfv_block
82func main(argc: i64, argv: *i64) -> i64