nx_quant_policy.nx

nx_quant_policy.nx -- per-layer mixed-precision quantization policy. Ships VRAM-track Q-003 per docs/VRAM_OPTIMIZATION_REALISTIC_TRACKING.md. Pure composition over the three quantizers already shipped: nx_quant_block.nx -- q4_0 (4-bit symmetric, 10.67x compression) nx_quant_block_q8.nx -- q8_0 (8-bit symmetric, 6.4x compression) nx_quant_q4k.nx -- q4_K (4-bit k-quants w/ per-group scales) The model loader consults this policy to decide WHICH quantizer to apply per-layer. Not every layer is equally sensitive to quantization loss; cheap-to-compress layers (FFN up/down/gate) go aggressive (q4_K), expensive-to-compress layers (Q/K/V projections, embeddings, final output) stay safer (q8_0 or passthrough at f16). Sensitivity heuristic literature: - Frantar 2022 _GPTQ_: per-channel scale, calibration-data-driven - Lin et al. 2023 _AWQ_: activation-aware -- protect outlier channels at higher precision - Xiao et al. 2023 _SmoothQuant_: pre-shift activation outliers into weights so weights can be quantized further without activation-side accuracy loss - ggml q-mix presets 2024: practitioner-tuned per-tensor-name defaults for Llama/Mistral/Qwen The substrate ships the POLICY TABLE structure + a default heuristic. Per-model calibration (the AWQ pass) is a separate workstream that fills the table with measured sensitivities. VRAM impact composition: * 70% of params at q4_K (4-bit + k-quant overhead): ~3.5x save * 25% of params at q8_0: ~1.4x save * 5% of params at f16 passthrough: ~0.5x save * Aggregate: ~3.0x save Per the bounded-loop cardinal: every loop here uses the LoopVerdict pattern from nx_loop.nx. genealogy_id: gptq_frantar_2022 + awq_lin_2023 + smoothquant_xiao_2023 + ggml_quant_mix_presets_2024