code wiki / _hdl_build / nx_qlayer.nx
nx_qlayer.nx
buildroot/runtime/_hdl_build/nx_qlayer.nx
about
nx_qlayer.nx -- the full decode LAYER: y = W . x, the operation an LLM does thousands of times
per token. Built honestly for the memory-bound regime: the weight matrix is PRE-QUANTIZED ONCE
(offline), and the HOT kernel streams ONLY the packed codes + per-block scales from DRAM -- the
exact deployment structure of ggml (weights quantized at load, dequant+dot in the inner loop).
This is the layer that, moving 3.5x less data at bounded quality, beats llama.cpp where it is
actually bottlenecked. The ENGINEER verifies the WHOLE layer output 1:1 vs the exact layer, and
the max per-row error is measured against the Council quality floor. license_tier: ORIGINAL
dependencies 1 imports · 2 importers
imports: nx_qmatvec.nx
imported by: nx_kquant.nxnx_qlayer_test.nx
structs
| none |
consts
| none |
functions
| 11 | func ql_blocks_per_row(c: i64) -> i64 { return (c + QMV_B - 1) / QMV_B } |
| 15 | func qlayer_quantize(w: *i64, r: i64, c: i64, qmax: i64, codes: *i64, scales: *i64) -> i64 |
| 37 | func qlayer_matvec(codes: *i64, scales: *i64, x: *i64, r: i64, c: i64, qmax: i64, y: *i64) -> i64 |
| 59 | func qlayer_matvec_exact(w: *i64, x: *i64, r: i64, c: i64, y: *i64) -> i64 |
| 72 | func qlayer_isqrt(n: i64) -> i64 called by 1: qlayer_l2_relerr_permil |
| 85 | func qlayer_l2_relerr_permil(yq: *i64, ye: *i64, r: i64) -> i64 |
| 105 | func qlayer_max_err_permil(yq: *i64, ye: *i64, r: i64) -> i64 |
| 121 | func qlayer_bytes_q(r: i64, c: i64, bits: i64) -> i64 { return (r * c * bits) / 8 + (r * ql_blocks_per_row(c)) * 2 } |
| 122 | func qlayer_bytes_ref16(r: i64, c: i64) -> i64 { return r * c * 2 } called by 1: main |