code wiki / (root) / nx_gen_lora_repr.nx

nx_gen_lora_repr.nx

buildroot/runtime/nx_gen_lora_repr.nx

8762 B179 linesdepth 9pulls 26 transitivereach 0 importersview sourcekind tooltopic gen
docsdependenciesstructsconstsfunctions

about

nx_gen_lora_repr.nx -- DECIDE the adapter representation by measuring it, not by arguing it. The hot matmul consumes Q8_0 blocks + per-block f16 scales. nx_lora_fold produces f32. So an adapted run has exactly two shapes, and they trade accuracy against per-step cost: A) FOLD-AND-REQUANTIZE — dequantize Q8_0, fold, requantize to Q8_0, run the normal kernel. One-time cost, zero per-step cost, hot loop untouched. This is what a "merged checkpoint" is. But it quantizes a weight that was ALREADY quantized once. B) RANK-FACTORED DELTA — keep the original Q8_0 blocks untouched and add scale * (h @ down^T) @ up^T in f32 at matmul time. The base carries NO new error. Costs 2*rank*(in+out) per token per step forever -- what sd.cpp pays. ★ BOTH ARE GRADED AGAINST THE EXACT f64 REFERENCE, NOT AGAINST EACH OTHER. Comparing A to B answers "do they differ", which is the question nobody asked; comparing each to the truth answers "which is right". This is the same two-way-comparison trap that made the oracle's quantization error look like ours for most of this lane. Usage: nx_gen_lora_repr <gguf> <lora.safetensors> [multiplier_milli] license_tier: ORIGINAL

dependencies 13 imports · 0 importers

nx_syscalls.nx nx_le.nx nx_f32.nx nx_f32_div.nx nx_f32_cvt.nx nx_f16.nx nx_strconv.nx nx_genfix.nx nx_genver.nx nx_genweights.nx nx_gen_lora_repr.nx

diagram shows first 10 each side; +3 more imports, +0 more importers in the complete lists below.

imports: nx_syscalls.nxnx_le.nxnx_f32.nxnx_f32_div.nxnx_f32_cvt.nxnx_f16.nxnx_strconv.nxnx_genfix.nxnx_genver.nxnx_genweights.nxnx_genblock.nxnx_q8_0_from_f32.nxnx_genlora.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main lr_puts sys_write nx_strconv_parse_i64 nx_ascii_is_digit sys_mmap nxa_die sys_write ↻ sys_exit nxa_lock_take nxa_lock_addr sys_write ↻ nxa_lock_give nxa_lock_addr ↻ nxa_report_overrun sys_write ↻ nxa_dump_printable sys_write ↻ nxa_dump_sizes sys_write ↻ nx_f32_div nx_f32_classify nx_f32_exp_field nx_f32_mant_field nx_f32_sign nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_i32_to_f32 nx_genfix_load _gf_path sys_mmap ↻ _gf_cpyz nx_genfix_root _gf_cpy sys_openat_rd sys_mmap ↻ sys_read sys_close br_strlen nx_gw_open

structs

none

consts

37const LR_IN: i64 = 3840
38const LR_OUT: i64 = 11520
39const LR_TOK: i64 = 768

functions

41func lr_puts(s: *u8) -> i64 {
called by 1: main calls 1: sys_write
56func lr_quant_q8(src: *u8, dst: *u8, in_dim: i64, out_dim: i64, scales: *i64) -> i64 {
76func main(argc: i64, argv: *i64) -> i64 {