nx_quant.nx
buildroot/runtime/nx_quant.nx
about
quant.nx -- quantization primitives for AI inference.
Packs / unpacks tensor data between fp16/fp32 and the
low-precision formats nxgguf supports (int8, int4, int2,
ternary, fp8 E4M3). These are the actual VRAM-saving
transforms that take a 32GB Llama 70B fp16 model down to
8GB int4 or 4GB int2.
Reference quantization schemes:
int8 SmoothQuant (Xiao et al. 2022)
int4 GPTQ (Frantar et al. 2022)
int4 AWQ (Lin et al. 2023)
GGUF k-quants (llama.cpp project)
BitNet b1.58 (Wang et al. 2024) -- ternary {-1, 0, +1}
v0.0.1 ships symmetric quantization with per-tensor scale.
k-quants (per-block scale + zero-point) follow in v0.1.0.
dependencies 1 imports · 1 importers
imports: nx_syscalls.nx
imported by: nx_parse.nx
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 26 | const K_MAGIC_2046: i64 = 2046 |
functions
| 49 | func fp32_from_parts(whole: i64, frac_num: i64, frac_digits: i64) -> i64 called by 1: parse_primary |
| 122 | func fp64_from_parts(whole: i64, frac_num: i64, frac_digits: i64) -> i64 called by 1: parse_primary |
| 247 | func fp64_to_fp32(bits64: i64) -> i64 called by 1: parse_stmt_let |
| 301 | func fp32_to_fp16(bits32: i64) -> i64 called by 1: main |
| 322 | func fp16_to_fp32(bits16: i64) -> i64 called by 1: main |
| 354 | func fp32_to_fp8e4m3(bits32: i64) -> i64 |
| 379 | func find_max_abs_fp32(data: *u8, n: i64) -> i64 |
| 401 | func quant_fp32_to_int8(value_bits: i64, scale_bits: i64) -> i64 |
| 417 | func dequant_int8_to_fp32(q: i64, scale_bits: i64) -> i64 |
| 435 | func pack_int4_pair(lo: i64, hi: i64) -> i64 |
| 442 | func unpack_int4_lo(byte: i64) -> i64 called by 1: main |
| 449 | func unpack_int4_hi(byte: i64) -> i64 called by 1: main |
| 457 | func pack_int4_array(in_vals: *u8, n: i64, out: *u8) -> i64 calls 1: pack_int4_pair |
| 473 | func pack_int2_quad(a: i64, b: i64, c: i64, d: i64) -> i64 called by 1: main |
| 477 | func unpack_int2_at(byte: i64, idx: i64) -> i64 called by 1: main |
| 482 | func ternary_encode(v: i64) -> i64 called by 1: main |
| 487 | func ternary_decode(b: i64) -> i64 called by 1: main |
| 495 | func main() -> i64 |