nx_q5_0_to_f32.nx
buildroot/runtime/nx_q5_0_to_f32.nx
about
nx_q5_0_to_f32.nx -- bits-up Q5_0 dequantization to IEEE 754 binary32.
Q5_0 block layout (per ggml v3 spec; clean-room from public docs):
18 bytes per block of 32 values:
bytes [0..2] d (f16 scale)
bytes [2..6] qh (32-bit packed high bits, LE)
bytes [6..22] qs (32 4-bit values, 2 per byte)
Dequant per ggml v3 spec (block_q5_0 in ggml-common.h):
For j in 0..15:
qs_byte = qs[j]
qs_low = qs_byte & 0x0F
qs_high = (qs_byte >> 4) & 0x0F
qh_bit_lo = (qh >> j) & 1 -- pairs with qs_low
qh_bit_hi = (qh >> (j + 16)) & 1 -- pairs with qs_high
q5_lo = qs_low | (qh_bit_lo << 4) // 0..31
q5_hi = qs_high | (qh_bit_hi << 4) // 0..31
value[j] = (q5_lo - 16) * d_f32
value[j+16] = (q5_hi - 16) * d_f32
CRITICAL: positions 0..15 use qs[j].low+qh_bit[j], positions 16..31
use qs[j].high+qh_bit[j+16]. The two halves of qs are interleaved
against DIFFERENT bit ranges of qh (not bits [i] for all i).
(Bug fix 2026-05-21 after live-fire on real Qwen2.5 surfaced
embed-magnitude blow-up due to wrong interleaving.)
genealogy_id: ggml_q5_0_canon
lineage_id: substrate_q5_0_to_f32_v1
dependencies 5 imports · 6 importers
imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_f32.nxnx_f32_cvt.nx
imported by: nx_f32_lazy_weight.nxnx_gguf_load_f32.nxnx_q5_0_fused_dot_gate.nxnx_q5_0_threaded_gate.nxnx_q5_0_to_f32_test.nxnx_q8_0_from_q5_0.nx
structs
| none |
consts
| 36 | const NX_Q5_0_OK: nx_int = 0 |
| 37 | const NX_Q5_0_ERR_BAD_DIM: nx_int = 1 |
| 38 | const NX_Q5_0_VPB: i64 = 32 // values per block (QK5_0) |
| 39 | const NX_Q5_0_BPB: i64 = 22 // bytes per block (d:2 + qh:4 + qs:16) |
functions
| 41 | func _nx_q5_0_block_to_f32(buf: *u8, base_off: i64, take: i64, |
| 81 | func nx_q5_0_to_f32(buf: *u8, base_off: i64, n_values: i64, |