nx_q8_matmul_micro.nx
buildroot/runtime/nx_q8_matmul_micro.nx
about
nx_q8_matmul_micro.nx -- DECISIVE micro-probe of the REAL production Q8_0
matmul path (nx_f32_lazy_weight_new_q8_0 + nx_f32_lazy_matmul ->
_lw_q8_0_matmul). The forward's MATMUL bucket is 72% of a decode token
(462ms), yet the ISOLATED nx_q8_0_simd_gate clocks this same kernel at
~20 GFLOP/s (=> lm_head should be ~14ms, but the forward pays ~143ms):
a 10x in-situ gap. This probe localizes it:
WARM same weight, m=1, matmul in a tight REP loop (weights become
cache-hot after pass 1) -> per-call us + GFLOP/s + bytes/s.
COLD N DISTINCT weight buffers (aggregate >> LLC), one matmul each,
NEVER reused -> the real-forward access pattern (every weight
read exactly once from DRAM).
If WARM ~ 20 GFLOP/s and COLD << that -> the forward is COLD-WEIGHT-
STREAMING bound (fix = fewer bytes or batch tokens, NOT a wider kernel).
If WARM is ALSO slow -> per-call overhead (mmap/pack/dispatch) or the
kernel itself. Shapes: W_gate (k=896,n=4864) + lm_head band (k=896,
n=32768). license_tier: ORIGINAL expect_exit: 0
dependencies 8 imports · 0 importers
imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_f32.nxnx_f32_cvt.nxnx_thread_pool.nxnx_f32_lazy_weight.nxnx_fmt.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 28 | const MK: i64 = 896 // hidden |
| 29 | const Q8B: i64 = 34 |
| 30 | const Q8V: i64 = 32 |
functions
| 32 | func mm_nl() -> i64 { fmt_puts("\n" as *u8); return 0 } |
| 33 | func mm_lcg(s: i64) -> i64 { var v: i64 = s * 1103515245 + 12345; v = v & 2147483647; return v } |
| 36 | func mm_weight(k: i64, n: i64, seed: i64) -> *u8 |
| 56 | func mm_report(tag: *u8, k: i64, n: i64, us: i64, calls: i64) -> i64 |
| 72 | func main() -> i64 |