nx_q8_0_from_f32.nx
buildroot/runtime/nx_q8_0_from_f32.nx
about
nx_q8_0_from_f32.nx -- quantize F32 -> Q8_0 blocks (the inverse of
nx_q8_0_to_f32), + a sovereign F32->F16 encoder (the ecosystem lacked one).
Used to re-quantize the Q6_K (and any F32/other) block weights to Q8_0 ONCE
AT LOAD so the forward runs the TRIVIAL Q8_0 SIMD dequant-dot (__f32_i8dot32,
10.2x) on ALL block weights. Near-lossless (Q8_0 8-bit >= Q6_K 6-bit); the
per-block absmax scale is the standard ggml Q8_0 quantizer.
Q8_0 block (34B): d(f16) + 32 int8. d = absmax/127; int8 = round(x/d).
lineage_id: q8_0_from_f32_v1
dependencies 3 imports · 2 importers
imports: nx_syscalls.nxnx_le.nxnx_f32_div.nx
imported by: nx_f32_llama_layer_lazy_load.nxnx_q8_0_from_f32_kat.nx
structs
| none |
consts
| none |
functions
| 18 | func nx_f32_to_f16(bits: i64) -> i64 called by 1: nx_q8_0_from_f32 |
| 44 | func _q8f_round_i(x: i64) -> i64 called by 1: nx_q8_0_from_f32 |
| 51 | func nx_q8_0_from_f32(f32_in: *i64, n_values: i64, out_q8: *u8) -> i64 |