nx_lowrank_kv_gate.nx
buildroot/runtime/nx_lowrank_kv_gate.nx
about
nx_lowrank_kv_gate.nx -- MEASURED Pareto gate: low-rank KV-cache compression via Frequent Directions.
module: nishi-core.ai.lowrank_kv capability: VRAM_REDUCTION (the quality-UP / VRAM-DOWN lever, NOT quant)
THESIS (operator 2026-06-16): innovate where math REDUCES VRAM while PRESERVING quality -- the opposite
of quantization (which shrinks quality AND size). The KV cache is up to ~70% of inference VRAM at long
context; attention logits depend on K^T K, so if a sketch B keeps ||K^T K - B^T B|| small we keep
attention fidelity while storing far fewer bytes. Frequent Directions (Liberty 2013, our sovereign
sketch_freq_directions.nx) gives exactly that with a PROVABLE bound: ||A^T A - B^T B||_F <= ||A-A_k||_F^2/(l-k).
THIS GATE MEASURES the Pareto by RUNNING (no self-grading):
PERSISTENT KV bytes: full = n*d*8 vs sketch = l*d*8 (CONSTANT in n -- the long-context win)
QUALITY (covariance fidelity): rel error ‰ of K^T K vs B^T B (lower = attention better preserved)
T1 sketch bytes are CONSTANT as context n grows (16 -> 64): VRAM does not explode with O(n).
T2 full KV bytes GROW with n (the baseline we beat).
T3 low-rank data: sketch fidelity error << full-rank data error (the metric is REAL, not vacuous --
FD can compress genuine low-rank structure but CANNOT fake-compress full-rank => neg-control).
T4 low-rank covariance error is below an absolute quality bar (quality PRESERVED, measured).
HONEST SCOPE: REFERENCE-SCALE (FD v1 caps d<=8, l<=4; real d=4096 needs FD v2 Jacobi/QR per its own
roadmap note). This proves the PRINCIPLE + the provable bound, sovereignly. Reuses FD (DRY). license_tier: ORIGINAL
dependencies 3 imports · 0 importers
imports: syscalls.nxsketch_freq_directions.nxsketch_types.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 26 | const KV_Q14: i64 = 16384 |
functions
| 29 | func kv_w(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 } called by 1: main |
| 30 | func kv_wn(v: i64) -> i64 called by 1: main |
| 43 | func kv_build_lowrank(K: *i64, n: i64, d: i64, rank: i64) -> i64 called by 1: kv_lowrank_err |
| 75 | func kv_build_fullrank(K: *i64, n: i64, d: i64) -> i64 called by 1: main |
| 89 | func kv_gram_full(K: *i64, n: i64, d: i64, G: *i64) -> i64 called by 1: kv_run_case |
| 106 | func kv_gram_sketch(fd: *FreqDir, d: i64, G: *i64) -> i64 |
| 124 | func kv_rel_err_permille(Gf: *i64, Gs: *i64, d: i64) -> i64 |
| 143 | func kv_run_case(K: *i64, n: i64, d: i64, l: i64, out_sketch_bytes: *i64) -> i64 called by 2: kv_lowrank_errmain calls 5: nx_fd_allocnx_fd_add_rowkv_gram_fullkv_gram_sketchkv_rel_err_permille |
| 163 | func kv_lowrank_err(n: i64, d: i64, l: i64) -> i64 |
| 170 | func main() -> i64 |