code wiki / _hdl_build / nx_kquant.nx

nx_kquant.nx

buildroot/runtime/_hdl_build/nx_kquant.nx

3006 B62 linesdepth 4pulls 4 transitivereach 1 importersview sourcekind library
docsdependenciesstructsconstsfunctions

about

nx_kquant.nx -- the QUALITY lever for quantization (quality is slightly more valuable than speed): ggml's K-quants beat naive Q4_0 mainly by FINER-GRAINED SCALES -- a big block shares ONE scale, so a single outlier weight inflates the step and wrecks the precision of every other weight in the block. Smaller sub-blocks ISOLATE outliers: only the sub-block holding the outlier pays, the rest stay sharp. This module makes the block granularity a PARAMETER, so the team can trade a little more scale storage for materially better quality -- another point on the quality/data frontier the Council balances. (Real NN weights are concentrated with occasional outliers, the exact case where this wins.) license_tier: ORIGINAL Refs: ggml Q4_K/Q6_K K-quants.

dependencies 2 imports · 1 importers

nx_qmatvec.nx nx_qlayer.nx nx_kquant.nx nx_kquant_test.nx

imports: nx_qmatvec.nxnx_qlayer.nx

imported by: nx_kquant_test.nx

structs

none

consts

none

functions

13func kq_blocks_per_row(c: i64, blk: i64) -> i64 { return (c + blk - 1) / blk }
16func kq_layer_quantize(w: *i64, r: i64, c: i64, qmax: i64, blk: i64, codes: *i64, scales: *i64) -> i64
37func kq_layer_matvec(codes: *i64, scales: *i64, x: *i64, r: i64, c: i64, qmax: i64, blk: i64, y: *i64) -> i64
called by 1: main calls 1: kq_blocks_per_row
59func kq_bytes(r: i64, c: i64, bits: i64, blk: i64) -> i64 { return (r * c * bits) / 8 + (r * kq_blocks_per_row(c, blk)) * 2 }
called by 1: main calls 1: kq_blocks_per_row
62func kq_bits_x100(bits: i64, blk: i64) -> i64 { return bits * 100 + (16 * 100) / blk }
called by 1: main