nx_q4k_matmul.nx
buildroot/runtime/nx_q4k_matmul.nx
about
nx_q4k_matmul.nx -- fused-dequant Q4_K row × column dot product.
Phase C foundation brick (conductor cardinal 2026-05-19).
Composes nx_dequant_iter.nx with the existing matmul pattern to
produce a row × column dot product WITHOUT materializing a full
dequantized weight tensor.
What the brick does:
* Caller holds the Q4_K-quantized weight buffer (e.g. raw bytes
from a GGUF file -- composes with nx_gguf_load_lazy)
* Caller holds the dense column vector (Q10 i64)
* This brick walks the row's Q4_K blocks, dequants each block's
256 values inline, multiplies by the corresponding column
entries, and accumulates the sum
* NO scratch buffer for dequantized weights; the substrate-side
RAM cost is the column vector + per-block iterator (~152 bytes)
Honest framing (per no-strawman-perf-comparisons cardinal):
- This brick proves the FUSED dequant+dot composition works and
produces bit-exact results vs the materialize-then-matmul path.
- Per-element throughput on qemu-RV64 is measured by the bench
below. This is NOT a claim about CUDA or llama.cpp -- those
systems have their own per-element costs that require running
them on the same hardware to compare honestly.
Output unit: dot product is returned in Q34 (sum of Q24 weight ×
Q10 column products). The Q4_K weight dequant is now Q24 (EXACT
super-scales -- Q10/Q14 underflow/flip sign on real trained-model
super-scales). Caller uses nx_q4km_q20_to_q10 to recover Q10
(>>24 round-half). Q34 is kept internally so successive matmuls can
compose without intermediate normalization rounding.
genealogy_id: ggml_q4k_dot_canon + standard_fused_dequant_pattern
lineage_id: substrate_q4k_matmul_v1_fused
dependencies 6 imports · 14 importers
diagram shows first 10 each side; +0 more imports, +4 more importers in the complete lists below.
imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_gguf.nxnx_gguf_load.nxnx_dequant_iter.nx
imported by: nx_f32_q4k_matmul.nxnx_q4k_dot_simd.nxnx_q4k_fused_vs_x4_gate.nxnx_q4k_gemm_mt.nxnx_q4k_ggml_kat.nxnx_q4k_linear.nxnx_q4k_linear_hp.nxnx_q4k_matmul_bench_test.nxnx_q4k_matmul_test.nxnx_q4k_real_gemm.nxnx_q4k_speed_bench.nxnx_qwen_hybrid_attn.nxnx_qwen_hybrid_ffn.nxnx_qwen_hybrid_qkv.nx
structs
| none |
consts
| 45 | const NX_Q4KM_OK: nx_int = 0 |
| 46 | const NX_Q4KM_ERR_BAD_DIM: nx_int = 1 |
| 47 | const NX_Q4KM_N_VERDICTS: nx_int = 2 |
| 132 | const NX_Q4KM_Q24: i64 = 16777216 // 2^24 (weight super-scale unit) |
| 133 | const NX_Q4KM_Q23: i64 = 8388608 // 2^23 (round-half bias) |
functions
| 49 | func nx_q4km_verdict_is_valid(v: nx_int) -> nx_int called by 1: main |
| 70 | func nx_q4k_dot_row_col(weight_buf: *u8, weight_super_off: i64, called by 11: mainmainmt_workermainkat_runnx_q4k_linear+5 calls 4: nx_q4k_iter_initnx_q4k_iter_scalenx_q4k_iter_minnx_le_read_u8 |
| 135 | func nx_q4km_q20_to_q10(dot_q34: i64) -> i64 |