code wiki / (root) / nx_quant_block_q8.nx

nx_quant_block_q8.nx

buildroot/runtime/nx_quant_block_q8.nx

11090 B301 linesdepth 3pulls 3 transitivereach 26 importersview sourcekind tooltopic quant
docsdependenciesstructsconstsfunctions

about

nx_quant_block_q8.nx -- 8-bit block quantization (q8_0 shape). Ships VRAM-track Q-001 per docs/VRAM_OPTIMIZATION_REALISTIC_TRACKING.md: the "safety floor" quantization that gives ~2x memory compression at <1% quality degradation -- well below human perception. Pair with q4_K (heavier compression, slight quality cost) per the mixed-precision Q-003 scenario. Per the sovereign-from-bits-up cardinal: pure NishiLang, no GGML dependency. Same block-layout contract as nx_quant_block.nx (the q4_0 sibling); the only difference is byte storage instead of nibble packing. Honest layout: * Block size: 32 values per block (same as q4_0) * Per-block scale: one i64 (Q-format multiplier) * Values: 8-bit signed bytes in [-127..127] (-128 unused; symmetric) * Storage per block: 8 (scale) + 32 (bytes) = 40 bytes * Dense storage: 32 i64 = 256 bytes * Compression: 256 / 40 = 6.4x (vs q4_0's 10.67x) Quality envelope (the honest measurement): * Per-value max rounding error = scale/2 = max_abs/254 * Relative error per value ≤ 1/254 ≈ 0.39% worst case * Mean error across uniform-distribution block ≈ 0.2% * Oracle round-trip uses eps_q10 = 12 (~1.2%) -- below human perception threshold (~1-2% for image content per LPIPS-calibrated psychophysics, Zhang 2018). Layout NOTE: we store signed bytes as offset u8 in [0..254] via the same +127 trick the q4 version uses for nibbles. -128 is excluded so the symmetric scale stays clean. genealogy_id: q8_0_ggml_2023 + jacob_kligys_2018_8bit_quant lineage_id: substrate_quant_block_q8_v1

dependencies 3 imports · 1 importers

nx_syscalls.nx nx_tier.nx nx_tensor.nx nx_quant_block_q8.nx nx_embedding.nx

imports: nx_syscalls.nxnx_tier.nxnx_tensor.nx

imported by: nx_embedding.nx

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap nx_qb8_alloc sys_mmap ↻ nx_qb8_quantize _qb8_abs _qb8_store_signed nx_qb8_dequantize _qb8_load_signed nx_qb8_compression_ratio_q

structs

77struct NxQuantBlockQ8

consts

46const NX_MAGIC_1024: i64 = 1024
47const NX_MAGIC_6553: i64 = 6553
48const NX_MAGIC_6350: i64 = 6350
50const NX_QB8_BLOCK_SIZE: nx_int = 32 // values per block (matches q4_0)
51const NX_QB8_BYTE_MAX: nx_int = 127 // [-127..+127]
52const NX_QB8_BYTE_OFFSET: nx_int = 127 // signed -> offset-u8 mapping
55const NX_QB8_BYTES_PER_BLOCK: nx_int = 40
59const NX_QB8_OK: nx_int = 0
60const NX_QB8_ERR_BAD_LEN: nx_int = 1 // n not multiple of BLOCK_SIZE
61const NX_QB8_ERR_BAD_DTYPE: nx_int = 2 // input not i64
62const NX_QB8_ERR_SHAPE_MISMATCH: nx_int = 3 // dequant target wrong shape
63const NX_QB8_N_VERDICTS: nx_int = 4
84const NX_QB8_STRUCT_BYTES: nx_int = 32 // 4 fields * 8

functions

65func nx_qb8_verdict_is_valid(v: nx_int) -> nx_int
86func nx_qb8_alloc(n_values: nx_int) -> *NxQuantBlockQ8
106func _qb8_abs(x: nx_int) -> nx_int
called by 1: nx_qb8_quantize
111func _qb8_store_signed(qb: *NxQuantBlockQ8, value_idx: nx_int, sb: nx_int) -> nx_int
called by 1: nx_qb8_quantize
120func _qb8_load_signed(qb: *NxQuantBlockQ8, value_idx: nx_int) -> nx_int
called by 1: nx_qb8_dequantize
131func nx_qb8_quantize(values: *i64, n: nx_int, qb: *NxQuantBlockQ8) -> nx_int
180func nx_qb8_dequantize(qb: *NxQuantBlockQ8, values_out: *i64, n: nx_int) -> nx_int
199func nx_qb8_compression_ratio_q10(qb: *NxQuantBlockQ8) -> nx_int
called by 1: main
208func nx_qb8_quantize_tensor(t: *NxTensor) -> *NxQuantBlockQ8
217func nx_qb8_dequantize_tensor(qb: *NxQuantBlockQ8, t: *NxTensor) -> nx_int
241func main() -> i64