code wiki / _hdl_build / nx_kquant.nx
nx_kquant.nx
buildroot/runtime/_hdl_build/nx_kquant.nx
about
nx_kquant.nx -- the QUALITY lever for quantization (quality is slightly more valuable than speed):
ggml's K-quants beat naive Q4_0 mainly by FINER-GRAINED SCALES -- a big block shares ONE scale,
so a single outlier weight inflates the step and wrecks the precision of every other weight in
the block. Smaller sub-blocks ISOLATE outliers: only the sub-block holding the outlier pays, the
rest stay sharp. This module makes the block granularity a PARAMETER, so the team can trade a
little more scale storage for materially better quality -- another point on the quality/data
frontier the Council balances. (Real NN weights are concentrated with occasional outliers, the
exact case where this wins.) license_tier: ORIGINAL Refs: ggml Q4_K/Q6_K K-quants.
dependencies 2 imports · 1 importers
imports: nx_qmatvec.nxnx_qlayer.nx
imported by: nx_kquant_test.nx
structs
| none |
consts
| none |
functions
| 13 | func kq_blocks_per_row(c: i64, blk: i64) -> i64 { return (c + blk - 1) / blk } |
| 16 | func kq_layer_quantize(w: *i64, r: i64, c: i64, qmax: i64, blk: i64, codes: *i64, scales: *i64) -> i64 |
| 37 | func kq_layer_matvec(codes: *i64, scales: *i64, x: *i64, r: i64, c: i64, qmax: i64, blk: i64, y: *i64) -> i64 |
| 59 | func kq_bytes(r: i64, c: i64, bits: i64, blk: i64) -> i64 { return (r * c * bits) / 8 + (r * kq_blocks_per_row(c, blk)) * 2 } |
| 62 | func kq_bits_x100(bits: i64, blk: i64) -> i64 { return bits * 100 + (16 * 100) / blk } called by 1: main |