nx_q5_0_threaded_gate.nx
buildroot/runtime/nx_q5_0_threaded_gate.nx
about
nx_q5_0_threaded_gate.nx -- THE DECISIVE test: does THREADING flip the
fused Q5_0 dequant-dot from a loss to a win?
Single-thread (nx_q5_0_fused_dot_gate) the fused kernel LOST 2x: compute-
bound (dequant overhead > MAC), memory not saturated. But at 16 cores the
F32 matmul saturates RAM bandwidth (16 x ~2GB/s > ~20-40GB/s), so reading
11.6x fewer bytes (Q5_0 3MB vs F32 34.9MB) could win THERE even with a
scalar dequant. Both paths POOL-threaded, SSE dot (__f32_mul/__f32_add),
same run => ratio is host-noise-tolerant. THIS decides whether quantized-
read is viable on our CPU substrate or needs SIMD-dequant intrinsics first.
NO forward changes.
Checks:
1 Q5_0-fused-pool == F32-pool (mmt_range), bit-exact
2 threaded ratio: fused vs F32 at FFN shape (report; not a hard floor --
this is a DECISION measurement, GREEN prints the verdict either way)
lineage_id: q5_0_threaded_gate_v1
dependencies 9 imports · 0 importers
imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_f32.nxnx_f32_cvt.nxnx_q5_0_to_f32.nxnx_thread_pool.nxnx_f32_matmul_t.nxnx_fmt.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| 36 | struct Q5Ctx |
consts
| 30 | const QK: i64 = 896 |
| 31 | const QN: i64 = 4864 |
| 32 | const QREPS: i64 = 120 |
| 33 | const Q5B: i64 = 22 |
| 34 | const Q5V: i64 = 32 |
| 45 | const Q5CTX_BYTES: i64 = 56 |
functions
| 47 | func q_lcg(s: i64) -> i64 { var v: i64 = s * 1103515245 + 12345; v = v & 2147483647; return v } |
| 49 | func q_fill_weight(w: *u8, seed: i64) -> i64 |
| 69 | func q_fill_a(p: *i64, count: i64, seed: i64) -> i64 |
| 78 | func q5_consts() -> *u8 |
| 101 | func q5_0_dot(qbuf: *u8, qoff: i64, Apk: *u8, k: i64, i8scr: *u8) -> i64 |
| 119 | func q5_task(ctx_i: i64) -> i64 |
| 135 | func q5_pool_matmul(pool: *NxThreadPool, qbuf: *u8, Apk: *u8, C: *i64, k: i64, n: i64) -> i64 |
| 162 | func q_nl() -> i64 { fmt_puts("\n" as *u8); return 0 } |
| 164 | func main() -> i64 |