quant.nx
buildroot/runtime/quant.nx
about
quant.nx -- quantization primitives for AI inference.
Packs / unpacks tensor data between fp16/fp32 and the
low-precision formats nxgguf supports (int8, int4, int2,
ternary, fp8 E4M3). These are the actual VRAM-saving
transforms that take a 32GB Llama 70B fp16 model down to
8GB int4 or 4GB int2.
Reference quantization schemes:
int8 SmoothQuant (Xiao et al. 2022)
int4 GPTQ (Frantar et al. 2022)
int4 AWQ (Lin et al. 2023)
GGUF k-quants (llama.cpp project)
BitNet b1.58 (Wang et al. 2024) -- ternary {-1, 0, +1}
v0.0.1 ships symmetric quantization with per-tensor scale.
k-quants (per-block scale + zero-point) follow in v0.1.0.
dependencies 1 imports · 2 importers
imports: syscalls.nx
imported by: fp32_parts_test.nxfp64_parts_test.nx
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| none |
functions
| 42 | func fp32_from_parts(whole: i64, frac_num: i64, frac_digits: i64) -> i64 {
called by 1: main |
| 115 | func fp64_from_parts(whole: i64, frac_num: i64, frac_digits: i64) -> i64 {
called by 1: main |
| 177 | func fp64_to_fp32(bits64: i64) -> i64 { |
| 231 | func fp32_to_fp16(bits32: i64) -> i64 {
called by 1: main |
| 252 | func fp16_to_fp32(bits16: i64) -> i64 {
called by 1: main |
| 284 | func fp32_to_fp8e4m3(bits32: i64) -> i64 { |
| 309 | func find_max_abs_fp32(data: *u8, n: i64) -> i64 { |
| 331 | func quant_fp32_to_int8(value_bits: i64, scale_bits: i64) -> i64 { |
| 347 | func dequant_int8_to_fp32(q: i64, scale_bits: i64) -> i64 { |
| 365 | func pack_int4_pair(lo: i64, hi: i64) -> i64 { |
| 372 | func unpack_int4_lo(byte: i64) -> i64 {
called by 1: main |
| 379 | func unpack_int4_hi(byte: i64) -> i64 {
called by 1: main |
| 387 | func pack_int4_array(in_vals: *u8, n: i64, out: *u8) -> i64 {
calls 1: pack_int4_pair |
| 403 | func pack_int2_quad(a: i64, b: i64, c: i64, d: i64) -> i64 {
called by 1: main |
| 407 | func unpack_int2_at(byte: i64, idx: i64) -> i64 {
called by 1: main |
| 412 | func ternary_encode(v: i64) -> i64 {
called by 1: main |
| 417 | func ternary_decode(b: i64) -> i64 {
called by 1: main |
| 425 | func main() -> i64 { |