nx_q5_k_to_f32.nx
buildroot/runtime/nx_q5_k_to_f32.nx
about
nx_q5_k_to_f32.nx -- bits-up Q5_K dequantization producing IEEE 754 binary32 outputs.
Q5_K = the (ggml-correct) Q4_K structure + a 5th bit per value from qh. Real Q4_K_M models store
attn_v / attn_output / ffn_down as Q5_K, so this is required to load them sovereignly.
ggml block_q5_K (176 bytes / 256 values):
[0..2) d f16 super-scale
[2..4) dmin f16 super-min
[4..16) scales 12 bytes = 8x(6-bit sc) + 8x(6-bit m) (get_scale_min_k4, same as Q4_K)
[16..48) qh 32 bytes = 1 high bit per value
[48..176) qs 128 bytes = 4 low bits per value
ggml dequantize_row_q5_K layout: 4 GROUPS of 64; group g feeds sub-block 2g (low nibble, 32 outputs)
and 2g+1 (high nibble, 32 outputs); the 5th bit is qh[l] tested with u1=1<<(2g) (low) / u2=1<<(2g+1)
(high). value = d*sc*(q4 + (bit?16:0)) - dmin*m (q5 in 0..31, min subtracted -- same form as Q4_K).
license_tier: ORIGINAL
dependencies 7 imports · 6 importers
imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_gguf.nxnx_gguf_load.nxnx_f32.nxnx_f32_cvt.nx
imported by: nx_f32_llama_layer_lazy_load.nxnx_q5_k_ggml_kat.nxnx_q5k_dot_row_col.nxnx_qwen_hybrid_attn.nxnx_qwen_hybrid_ffn.nxnx_qwen_hybrid_qkv.nx
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 24 | const NX_Q5K_OK: nx_int = 0 |
| 25 | const NX_Q5_K_VPB: i64 = 256 |
| 26 | const NX_Q5_K_BPB: i64 = 176 |
functions
| 28 | func nx_q5k_block_to_f32(buf: *u8, super_off: i64, n_values: i64, out_f32: *i64) -> nx_int |
| 105 | func nx_q5_k_to_f32(buf: *u8, base_off: i64, n_values: i64, out_f32: *i64) -> nx_int |
| 122 | func main() -> i64 |