nx_genlora.nx

nx_genlora.nx -- apply LoRA adapters to a base weight. Sovereign, model-agnostic LoRA support. W' = W + (alpha / rank) * multiplier * (up @ down) read out of sd.cpp's lora.hpp, not guessed: `scale_value = alpha / rank; scale_value *= multiplier`. With no `.alpha` tensor, alpha defaults to rank, so the scale is the multiplier alone. ⚠TWO THINGS VARY INDEPENDENTLY, AND BOTH MUST BE HANDLED: 1. THE NAMING CONVENTION. base model.diffusion_model.layers.0.attention.qkv.weight kohya lora_unet_layers_0_attention_qkv.lora_down.weight (flattened, underscores) peft diffusion_model.layers.0.attention.to_q.lora_A.weight (dotted, A=down B=up) 2. WHETHER THE ADAPTER TRAINED THE **FUSED** MODULE OR THE UNFUSED ONES. This is the part that bites. The Z-Image checkpoint stores ONE fused `attention.qkv.weight` [11520, 3840]. The kohya adapter here matches it directly. But all NINE peft adapters in the same folder train `to_q` / `to_k` / `to_v` SEPARATELY -- there is no `attention.qkv` key in any of them. A name-only lookup therefore finds nothing, folds nothing, and returns "this adapter does not cover this layer" for EVERY layer of EVERY peft adapter, with no error anywhere. ★★★★★ AN ADAPTER THAT MATCHES NO KEYS IS INDISTINGUISHABLE FROM ONE THAT IS WORKING WEAKLY. ⇒ a fused base weight must be folded SLICE-WISE from the unfused adapter modules: to_q -> rows [0, out/3) to_k -> [out/3, 2out/3) to_v -> [2out/3, out) ★ MEASURED CAVEAT worth more than the code: on Z-Image layer 0 the kohya adapter moves the activation by only 0.86% while the oracle's own Q8_0 error is 0.6% median. Fitting the scale by least squares against the oracle recovered 0.1725 where alpha/rank is 0.1667 -- consistent, but INSIDE THE NOISE. ★★★★★ YOU CANNOT VALIDATE AN ADAPTER ON A LAYER WHERE THE ADAPTER BARELY ACTS; grade against an EXACT reference (nx_gen_lora_verify does). WHY FOLD RATHER THAN APPLY AT RUNTIME: sd.cpp keeps the adapter separate and pays 2*rank*(in+out)*tokens every step. Folding pays 2*rank*in*out ONCE and leaves the hot matmul byte-identical to the un-adapted path -- so an adapter costs nothing per step and cannot slow down or perturb the kernel. license_tier: ORIGINAL