code wiki / (root) / nx_q4k_matmul_rate.nx

nx_q4k_matmul_rate.nx

buildroot/runtime/nx_q4k_matmul_rate.nx

3336 B74 linesdepth 9pulls 22 transitivereach 0 importersview sourcekind tooltopic q4k
docsdependenciesstructsconstsfunctions

about

nx_q4k_matmul_rate.nx -- isolate the SSE Q4_K matmul rate (no 491MB model load) to pinpoint why the LM forward is slow (408s/8tok). Times nx_f32_q4k_matmul on a real LM size (W_gate: m=8, k=896, n=4864) on DUMMY Q4_K bytes (timing is layout-correct regardless of byte values). MFLOP/s tells us: ~30 = matmul is OVERHEAD-bound (dequant + i64-boxing + indexing dominate; scalar SSE can't help) -> packed SIMD/codegen is the lever; ~300 = matmul is fast and the forward's slowness is elsewhere (attention/softmax/rmsnorm).

dependencies 2 imports · 0 importers

nx_syscalls.nx nx_f32_q4k_matmul.nx nx_q4k_matmul_rate.nx

imports: nx_syscalls.nxnx_f32_q4k_matmul.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main bn_puts sys_write sys_mmap sys_now_ms sys_mmap ↻ sys_clock_gettime_mono nx_f32_q4k_matmul fq4m_rows sys_mmap ↻ _fq4m_pack_a nx_q4k_to_f32_packed nx_q4k_block_to_f32_packed nx_le_read_u16 nx_f16_to_f32 nx_le_read_u8 nx_i32_to_f32 _q4kp_st4 sys_munmap bn_putn sys_write ↻ sys_mmap ↻ sys_exit

structs

none

consts

none

functions

9func bn_puts(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 }
called by 1: main calls 1: sys_write
10func bn_putn(v: i64) -> i64
called by 1: main calls 2: sys_writesys_mmap
21func main() -> i64
calls 6: bn_putssys_mmapsys_now_msnx_f32_q4k_matmulbn_putnsys_exit