code wiki / (root) / nx_dequant_iter_bench.nx

nx_dequant_iter_bench.nx

buildroot/runtime/nx_dequant_iter_bench.nx

1102 B25 linesdepth 0pulls 0 transitivereach 1 importersview sourcekind benchtopic dequant
docsdependenciesstructsconstsfunctions

about

nx_dequant_iter_bench.nx -- partial-access throughput comparison. Measures: when a caller needs only K of 256 values per Q4_K block, what's the substrate-side cost? Two paths timed: A) WHOLE-BLOCK PATH: nx_gguf_dequant_q4_k materializes all 256 values for every block, even if caller throws 256-K of them away. B) ITER PATH: nx_q4k_iter_init once per block + K calls to nx_q4k_iter_value for just the K needed values. Workload: N blocks * K reads each, for K in {1, 8, 32, 256}. HONEST FRAMING: - These numbers are on qemu-riscv64 emulation; native silicon would be faster. - The comparison is BETWEEN OUR OWN two paths, not against any other framework. No "Nx faster than CUDA / llama.cpp" claim. - At K=256 the iter path is expected to be roughly equivalent to (or slightly slower than) the whole-block path -- both do the same total work. At K<256 the iter path skips work.

dependencies 0 imports · 1 importers

nx_dequant_iter_bench.nx nx_dequant_iter_bench_test.nx

imports: none

imported by: nx_dequant_iter_bench_test.nx

structs

none

consts

24const NX_DIB_N_BLOCKS: i64 = 32 // blocks per measurement
25const NX_DIB_K_RUNS: i64 = 16 // repetitions per K

functions

none