code wiki / _hdl_build / nx_quality_balance.nx
nx_quality_balance.nx
buildroot/runtime/_hdl_build/nx_quality_balance.nx
about
nx_quality_balance.nx -- BALANCE quality against speed (the operator's law: always balance
quality and quantity, with QUALITY SLIGHTLY MORE VALUABLE than raw speed). Cutting bytes
(lower quantization) buys speed in the memory-bound regime but costs QUALITY (perplexity rises).
"Just go Q2 for 1.8x" is wrong -- Q2_K is +8.82% perplexity. This module picks the operating
point that maximizes a quality-weighted score under a hard QUALITY FLOOR (rule: never strip
quality for speed). Data-driven: the quality + size numbers are measured config, not magic.
RACI: the COUNCIL governs the POLICY here (the quality floor + the quality:speed weighting --
the value judgment). The ENGINEER supplies the measured numbers (nx_engineer_profile). The
BUILDER consumes qb_select to choose the operating point (nx_builder_quant). One organ per job.
Real llama.cpp data (Llama-3.1-8B class), perplexity delta to fp16 as PERMIL (pct*10), and bits
per weight *10: fp16(0,160) Q8_0(1,85) Q6_K(2,66) Q5_K_M(4,55) Q4_K_M(12,45) Q3_K_M(47,34)
Q2_K(88,26). Refs: arxiv 2601.14277 unified quant eval; llama.cpp perplexity discussion #406.
license_tier: ORIGINAL
dependencies 1 imports · 1 importers
imports: nx_syscalls.nx
imported by: nx_builder_quant.nx
structs
| none |
consts
| 20 | const QB_W_QUALITY: i64 = 11 // quality weight (slightly higher than speed) |
| 21 | const QB_W_SPEED: i64 = 9 // speed weight |
| 22 | const QB_QUALITY_FLOOR: i64 = 20 // max acceptable perplexity loss, PERMIL (2.0%); below = reject |
| 23 | const QB_BITS_FP16: i64 = 160 // bits*10 of the fp16 reference (the size baseline) |
| 24 | const QB_BITS_MIN: i64 = 26 // bits*10 of the smallest option (Q2_K) -- speed normalisation span |
functions
| 27 | func qb_quality_score(loss_permil: i64) -> i64 called by 1: qb_combined |
| 35 | func qb_speed_score(bits10: i64) -> i64 called by 1: qb_combined |
| 45 | func qb_combined(loss_permil: i64, bits10: i64) -> i64 |
| 54 | func council_quality_admit(loss_permil: i64, floor: i64) -> i64 |
| 63 | func qb_select(n: i64, loss: *i64, bits10: *i64, floor: i64) -> i64 |
| 83 | func qb_select_speed_only(n: i64, bits10: *i64) -> i64 called by 1: main |