nx_quant_block_q8.nx
buildroot/runtime/nx_quant_block_q8.nx
about
nx_quant_block_q8.nx -- 8-bit block quantization (q8_0 shape).
Ships VRAM-track Q-001 per docs/VRAM_OPTIMIZATION_REALISTIC_TRACKING.md:
the "safety floor" quantization that gives ~2x memory compression
at <1% quality degradation -- well below human perception. Pair
with q4_K (heavier compression, slight quality cost) per the
mixed-precision Q-003 scenario.
Per the sovereign-from-bits-up cardinal: pure NishiLang, no GGML
dependency. Same block-layout contract as nx_quant_block.nx (the
q4_0 sibling); the only difference is byte storage instead of
nibble packing.
Honest layout:
* Block size: 32 values per block (same as q4_0)
* Per-block scale: one i64 (Q-format multiplier)
* Values: 8-bit signed bytes in [-127..127] (-128 unused; symmetric)
* Storage per block: 8 (scale) + 32 (bytes) = 40 bytes
* Dense storage: 32 i64 = 256 bytes
* Compression: 256 / 40 = 6.4x (vs q4_0's 10.67x)
Quality envelope (the honest measurement):
* Per-value max rounding error = scale/2 = max_abs/254
* Relative error per value ≤ 1/254 ≈ 0.39% worst case
* Mean error across uniform-distribution block ≈ 0.2%
* Oracle round-trip uses eps_q10 = 12 (~1.2%) -- below human
perception threshold (~1-2% for image content per
LPIPS-calibrated psychophysics, Zhang 2018).
Layout NOTE: we store signed bytes as offset u8 in [0..254] via
the same +127 trick the q4 version uses for nibbles. -128 is
excluded so the symmetric scale stays clean.
genealogy_id: q8_0_ggml_2023 + jacob_kligys_2018_8bit_quant
lineage_id: substrate_quant_block_q8_v1
dependencies 3 imports · 1 importers
imports: nx_syscalls.nxnx_tier.nxnx_tensor.nx
imported by: nx_embedding.nx
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| 77 | struct NxQuantBlockQ8 |
consts
| 46 | const NX_MAGIC_1024: i64 = 1024 |
| 47 | const NX_MAGIC_6553: i64 = 6553 |
| 48 | const NX_MAGIC_6350: i64 = 6350 |
| 50 | const NX_QB8_BLOCK_SIZE: nx_int = 32 // values per block (matches q4_0) |
| 51 | const NX_QB8_BYTE_MAX: nx_int = 127 // [-127..+127] |
| 52 | const NX_QB8_BYTE_OFFSET: nx_int = 127 // signed -> offset-u8 mapping |
| 55 | const NX_QB8_BYTES_PER_BLOCK: nx_int = 40 |
| 59 | const NX_QB8_OK: nx_int = 0 |
| 60 | const NX_QB8_ERR_BAD_LEN: nx_int = 1 // n not multiple of BLOCK_SIZE |
| 61 | const NX_QB8_ERR_BAD_DTYPE: nx_int = 2 // input not i64 |
| 62 | const NX_QB8_ERR_SHAPE_MISMATCH: nx_int = 3 // dequant target wrong shape |
| 63 | const NX_QB8_N_VERDICTS: nx_int = 4 |
| 84 | const NX_QB8_STRUCT_BYTES: nx_int = 32 // 4 fields * 8 |
functions
| 65 | func nx_qb8_verdict_is_valid(v: nx_int) -> nx_int |
| 86 | func nx_qb8_alloc(n_values: nx_int) -> *NxQuantBlockQ8 |
| 106 | func _qb8_abs(x: nx_int) -> nx_int called by 1: nx_qb8_quantize |
| 111 | func _qb8_store_signed(qb: *NxQuantBlockQ8, value_idx: nx_int, sb: nx_int) -> nx_int called by 1: nx_qb8_quantize |
| 120 | func _qb8_load_signed(qb: *NxQuantBlockQ8, value_idx: nx_int) -> nx_int called by 1: nx_qb8_dequantize |
| 131 | func nx_qb8_quantize(values: *i64, n: nx_int, qb: *NxQuantBlockQ8) -> nx_int |
| 180 | func nx_qb8_dequantize(qb: *NxQuantBlockQ8, values_out: *i64, n: nx_int) -> nx_int |
| 199 | func nx_qb8_compression_ratio_q10(qb: *NxQuantBlockQ8) -> nx_int called by 1: main |
| 208 | func nx_qb8_quantize_tensor(t: *NxTensor) -> *NxQuantBlockQ8 |
| 217 | func nx_qb8_dequantize_tensor(qb: *NxQuantBlockQ8, t: *NxTensor) -> nx_int |
| 241 | func main() -> i64 |