code wiki / (root) / nx_q8_matmul_micro.nx

nx_q8_matmul_micro.nx

buildroot/runtime/nx_q8_matmul_micro.nx

6130 B146 linesdepth 10pulls 30 transitivereach 0 importersview sourcekind tooltopic q8
docsdependenciesstructsconstsfunctions

about

nx_q8_matmul_micro.nx -- DECISIVE micro-probe of the REAL production Q8_0 matmul path (nx_f32_lazy_weight_new_q8_0 + nx_f32_lazy_matmul -> _lw_q8_0_matmul). The forward's MATMUL bucket is 72% of a decode token (462ms), yet the ISOLATED nx_q8_0_simd_gate clocks this same kernel at ~20 GFLOP/s (=> lm_head should be ~14ms, but the forward pays ~143ms): a 10x in-situ gap. This probe localizes it: WARM same weight, m=1, matmul in a tight REP loop (weights become cache-hot after pass 1) -> per-call us + GFLOP/s + bytes/s. COLD N DISTINCT weight buffers (aggregate >> LLC), one matmul each, NEVER reused -> the real-forward access pattern (every weight read exactly once from DRAM). If WARM ~ 20 GFLOP/s and COLD << that -> the forward is COLD-WEIGHT- STREAMING bound (fix = fewer bytes or batch tokens, NOT a wider kernel). If WARM is ALSO slow -> per-call overhead (mmap/pack/dispatch) or the kernel itself. Shapes: W_gate (k=896,n=4864) + lm_head band (k=896, n=32768). license_tier: ORIGINAL expect_exit: 0

dependencies 8 imports · 0 importers

nx_syscalls.nx nx_tier.nx nx_le.nx nx_f32.nx nx_f32_cvt.nx nx_thread_pool.nx nx_f32_lazy_weight.nx nx_fmt.nx nx_q8_matmul_micro.nx

imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_f32.nxnx_f32_cvt.nxnx_thread_pool.nxnx_f32_lazy_weight.nxnx_fmt.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap mm_lcg nx_i32_to_f32 nx_lw_shared_pool nx_pool_is_native mm_weight sys_mmap ↻ mm_lcg ↻ nx_f32_lazy_weight_new_q8_ sys_mmap ↻ _lw_q8_0_matmul_pool_force nx_lw_shared_pool ↻ _q8st_apk sys_mmap ↻ _fq4m_pack_a _q8mt_ctxs sys_mmap ↻ nx_pool_n_completed nx_atom_load_i64 nx_pool_submit nx_atom_faa_i64 nx_chan_send nx_chan_try_send _nx_chan_cell nx_thread_yield _pool_futex_wake_all sys_futex_wake nx_pool_wait nx_atom_load_i64 ↻ nx_thread_yield ↻ nx_atom_store_i64 _pool_futex_wait sys_futex_wait sys_now_us sys_mmap ↻ sys_clock_gettime_mono mm_report fmt_puts fmt_puts_fd

structs

none

consts

28const MK: i64 = 896 // hidden
29const Q8B: i64 = 34
30const Q8V: i64 = 32

functions

32func mm_nl() -> i64 { fmt_puts("\n" as *u8); return 0 }
called by 2: mm_reportmain calls 1: fmt_puts
33func mm_lcg(s: i64) -> i64 { var v: i64 = s * 1103515245 + 12345; v = v & 2147483647; return v }
called by 2: mm_weightmain
36func mm_weight(k: i64, n: i64, seed: i64) -> *u8
called by 1: main calls 2: sys_mmapmm_lcg
56func mm_report(tag: *u8, k: i64, n: i64, us: i64, calls: i64) -> i64
called by 1: main calls 3: fmt_putsfmt_putnmm_nl
72func main() -> i64