nx_dequant_iter_bench.nx source
↩ module page · 25 lines · 1102 B
1// nx_dequant_iter_bench.nx -- partial-access throughput comparison.
2//
3// Measures: when a caller needs only K of 256 values per Q4_K block,
4// what's the substrate-side cost?
5//
6// Two paths timed:
7// A) WHOLE-BLOCK PATH: nx_gguf_dequant_q4_k materializes all 256
8// values for every block, even if caller throws 256-K of them
9// away.
10// B) ITER PATH: nx_q4k_iter_init once per block + K calls to
11// nx_q4k_iter_value for just the K needed values.
12//
13// Workload: N blocks * K reads each, for K in {1, 8, 32, 256}.
14//
15// HONEST FRAMING:
16// - These numbers are on qemu-riscv64 emulation; native silicon
17// would be faster.
18// - The comparison is BETWEEN OUR OWN two paths, not against any
19// other framework. No "Nx faster than CUDA / llama.cpp" claim.
20// - At K=256 the iter path is expected to be roughly equivalent
21// to (or slightly slower than) the whole-block path -- both do
22// the same total work. At K<256 the iter path skips work.
23
24const NX_DIB_N_BLOCKS: i64 = 32 // blocks per measurement
25const NX_DIB_K_RUNS: i64 = 16 // repetitions per K