code wiki / (root) / nx_q4k_matmul.nx

nx_q4k_matmul.nx

buildroot/runtime/nx_q4k_matmul.nx

6059 B140 linesdepth 7pulls 9 transitivereach 82 importersview sourcekind librarytopic q4k
docsdependenciesstructsconstsfunctions

about

nx_q4k_matmul.nx -- fused-dequant Q4_K row × column dot product. Phase C foundation brick (conductor cardinal 2026-05-19). Composes nx_dequant_iter.nx with the existing matmul pattern to produce a row × column dot product WITHOUT materializing a full dequantized weight tensor. What the brick does: * Caller holds the Q4_K-quantized weight buffer (e.g. raw bytes from a GGUF file -- composes with nx_gguf_load_lazy) * Caller holds the dense column vector (Q10 i64) * This brick walks the row's Q4_K blocks, dequants each block's 256 values inline, multiplies by the corresponding column entries, and accumulates the sum * NO scratch buffer for dequantized weights; the substrate-side RAM cost is the column vector + per-block iterator (~152 bytes) Honest framing (per no-strawman-perf-comparisons cardinal): - This brick proves the FUSED dequant+dot composition works and produces bit-exact results vs the materialize-then-matmul path. - Per-element throughput on qemu-RV64 is measured by the bench below. This is NOT a claim about CUDA or llama.cpp -- those systems have their own per-element costs that require running them on the same hardware to compare honestly. Output unit: dot product is returned in Q34 (sum of Q24 weight × Q10 column products). The Q4_K weight dequant is now Q24 (EXACT super-scales -- Q10/Q14 underflow/flip sign on real trained-model super-scales). Caller uses nx_q4km_q20_to_q10 to recover Q10 (>>24 round-half). Q34 is kept internally so successive matmuls can compose without intermediate normalization rounding. genealogy_id: ggml_q4k_dot_canon + standard_fused_dequant_pattern lineage_id: substrate_q4k_matmul_v1_fused

dependencies 6 imports · 14 importers

nx_syscalls.nx nx_tier.nx nx_le.nx nx_gguf.nx nx_gguf_load.nx nx_dequant_iter.nx nx_q4k_matmul.nx nx_f32_q4k_matmul.nx nx_q4k_dot_simd.nx nx_q4k_fused_vs_x4_gate.nx nx_q4k_gemm_mt.nx nx_q4k_ggml_kat.nx nx_q4k_linear.nx nx_q4k_linear_hp.nx nx_q4k_matmul_bench_test.nx nx_q4k_matmul_test.nx nx_q4k_real_gemm.nx

diagram shows first 10 each side; +0 more imports, +4 more importers in the complete lists below.

imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_gguf.nxnx_gguf_load.nxnx_dequant_iter.nx

imported by: nx_f32_q4k_matmul.nxnx_q4k_dot_simd.nxnx_q4k_fused_vs_x4_gate.nxnx_q4k_gemm_mt.nxnx_q4k_ggml_kat.nxnx_q4k_linear.nxnx_q4k_linear_hp.nxnx_q4k_matmul_bench_test.nxnx_q4k_matmul_test.nxnx_q4k_real_gemm.nxnx_q4k_speed_bench.nxnx_qwen_hybrid_attn.nxnx_qwen_hybrid_ffn.nxnx_qwen_hybrid_qkv.nx

structs

none

consts

45const NX_Q4KM_OK: nx_int = 0
46const NX_Q4KM_ERR_BAD_DIM: nx_int = 1
47const NX_Q4KM_N_VERDICTS: nx_int = 2
132const NX_Q4KM_Q24: i64 = 16777216 // 2^24 (weight super-scale unit)
133const NX_Q4KM_Q23: i64 = 8388608 // 2^23 (round-half bias)

functions

49func nx_q4km_verdict_is_valid(v: nx_int) -> nx_int
called by 1: main
70func nx_q4k_dot_row_col(weight_buf: *u8, weight_super_off: i64,
135func nx_q4km_q20_to_q10(dot_q34: i64) -> i64