nx_dequant_iter_bench.nx
buildroot/runtime/nx_dequant_iter_bench.nx
about
nx_dequant_iter_bench.nx -- partial-access throughput comparison.
Measures: when a caller needs only K of 256 values per Q4_K block,
what's the substrate-side cost?
Two paths timed:
A) WHOLE-BLOCK PATH: nx_gguf_dequant_q4_k materializes all 256
values for every block, even if caller throws 256-K of them
away.
B) ITER PATH: nx_q4k_iter_init once per block + K calls to
nx_q4k_iter_value for just the K needed values.
Workload: N blocks * K reads each, for K in {1, 8, 32, 256}.
HONEST FRAMING:
- These numbers are on qemu-riscv64 emulation; native silicon
would be faster.
- The comparison is BETWEEN OUR OWN two paths, not against any
other framework. No "Nx faster than CUDA / llama.cpp" claim.
- At K=256 the iter path is expected to be roughly equivalent
to (or slightly slower than) the whole-block path -- both do
the same total work. At K<256 the iter path skips work.
dependencies 0 imports · 1 importers
imports: none
imported by: nx_dequant_iter_bench_test.nx
structs
| none |
consts
| 24 | const NX_DIB_N_BLOCKS: i64 = 32 // blocks per measurement |
| 25 | const NX_DIB_K_RUNS: i64 = 16 // repetitions per K |
functions
| none |