nx_gen_lora_repr.nx
buildroot/runtime/nx_gen_lora_repr.nx
about
nx_gen_lora_repr.nx -- DECIDE the adapter representation by measuring it, not by arguing it.
The hot matmul consumes Q8_0 blocks + per-block f16 scales. nx_lora_fold produces f32. So an
adapted run has exactly two shapes, and they trade accuracy against per-step cost:
A) FOLD-AND-REQUANTIZE — dequantize Q8_0, fold, requantize to Q8_0, run the normal kernel.
One-time cost, zero per-step cost, hot loop untouched. This is what a "merged checkpoint"
is. But it quantizes a weight that was ALREADY quantized once.
B) RANK-FACTORED DELTA — keep the original Q8_0 blocks untouched and add
scale * (h @ down^T) @ up^T
in f32 at matmul time. The base carries NO new error. Costs 2*rank*(in+out) per token per
step forever -- what sd.cpp pays.
★ BOTH ARE GRADED AGAINST THE EXACT f64 REFERENCE, NOT AGAINST EACH OTHER. Comparing A to B
answers "do they differ", which is the question nobody asked; comparing each to the truth answers
"which is right". This is the same two-way-comparison trap that made the oracle's quantization
error look like ours for most of this lane.
Usage: nx_gen_lora_repr <gguf> <lora.safetensors> [multiplier_milli]
license_tier: ORIGINAL
dependencies 13 imports · 0 importers
diagram shows first 10 each side; +3 more imports, +0 more importers in the complete lists below.
imports: nx_syscalls.nxnx_le.nxnx_f32.nxnx_f32_div.nxnx_f32_cvt.nxnx_f16.nxnx_strconv.nxnx_genfix.nxnx_genver.nxnx_genweights.nxnx_genblock.nxnx_q8_0_from_f32.nxnx_genlora.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 37 | const LR_IN: i64 = 3840 |
| 38 | const LR_OUT: i64 = 11520 |
| 39 | const LR_TOK: i64 = 768 |
functions
| 41 | func lr_puts(s: *u8) -> i64 { |
| 56 | func lr_quant_q8(src: *u8, dst: *u8, in_dim: i64, out_dim: i64, scales: *i64) -> i64 { |
| 76 | func main(argc: i64, argv: *i64) -> i64 { |