code wiki / _hdl_build / nx_lowrank_train.nx
nx_lowrank_train.nx
buildroot/runtime/_hdl_build/nx_lowrank_train.nx
about
nx_lowrank_train.nx -- M2: does a LOW-RANK bottleneck IMPROVE quality (not just preserve it)? The MLA/
inductive-bias thesis (operator: "quality UP, VRAM down -- not just quant shrinking quality"). Trains
models on a NOISY rank-2 target, scored on a CLEAN held-out test set, swept over many SEEDS (robustness)
AND over bottleneck RANK (the U-curve -- guards the over-claim "smaller is always better"):
FULL: y = W x (W is D x D = 64 params) -- can overfit the noise
LOWRANK: y = Wu (Wd x) (Wd r x D, Wu D x r = 2*r*D params, the rank-r bottleneck)
CLAIM (precise, not over-claimed): the OPTIMAL bottleneck rank MATCHES the data's true rank R0=2 -- too
small (r=1) UNDERFITS, too large (r=4) OVERFITS like full, and at r=R0 it beats full at HALF the VRAM.
Sovereign autograd (nx_tgrad_core tg_* + AdamW ad_step; the proven _t4 recipe). nx_cc UNTOUCHED (operator:
it's the parallel effort). license_tier: ORIGINAL
dependencies 2 imports · 0 importers
imports: nx_syscalls.nxnx_tgrad_core.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 13 | const M2_MAGIC_1103515245: i64 = 1103515245 |
| 14 | const M2_MAGIC_12345: i64 = 12345 |
| 15 | const M2_MAGIC_2048: i64 = 2048 |
| 16 | const M2_MAGIC_4096: i64 = 4096 |
| 17 | const M2_MAGIC_20480: i64 = 20480 |
| 18 | const M2_MAGIC_16384: i64 = 16384 |
| 19 | const M2_MAGIC_100000: i64 = 100000 |
| 20 | const M2_MAGIC_1000003: i64 = 1000003 |
| 21 | const M2_MAGIC_7919: i64 = 7919 |
| 23 | const M2_D: i64 = 8 |
| 24 | const M2_R0: i64 = 2 // TRUE target rank |
| 25 | const M2_R: i64 = 2 // low-rank model bottleneck (matched to R0) |
| 26 | const M2_RMAX: i64 = 4 // max bottleneck rank in the sweep (buffer sizing) |
| 27 | const M2_NTR: i64 = 12 // train samples |
| 28 | const M2_NTE: i64 = 8 // test samples |
| 29 | const M2_EPOCHS: i64 = 500 |
| 30 | const M2_NSEED: i64 = 9 // seed sweep -- robustness, not one lucky draw |
functions
| 32 | func m2_puts(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 } |
| 33 | func m2_putn(v: i64) -> i64 |
| 43 | func m2_lcg(st: *i64) -> i64 { st[0] = st[0] * M2_MAGIC_1103515245 + M2_MAGIC_12345; return (st[0] >> 20) & 0xfff } |
| 44 | func m2_rf(st: *i64) -> i64 { return tg_q(m2_lcg(st) - M2_MAGIC_2048, M2_MAGIC_4096) } // ~[-0.5, 0.5] |
| 45 | func m2_rsmall(st: *i64) -> i64 { return tg_q(m2_lcg(st) - M2_MAGIC_2048, M2_MAGIC_20480) } // ~[-0.1, 0.1] (weight init) |
| 48 | func m2_apply(x: *i64, U: *i64, V: *i64, y: *i64) -> i64 |
| 66 | func m2_gen(X: *i64, Y: *i64, n: i64, U: *i64, V: *i64, st: *i64, noisy: i64) -> i64 |
| 81 | func m2_train_full(tape: *i64, nb: *i64, arena: *i64, ab: *i64, X: *i64, Y: *i64, W: *i64, mW: *i64, vW: *i64) -> i64 |
| 107 | func m2_train_low(tape: *i64, nb: *i64, arena: *i64, ab: *i64, X: *i64, Y: *i64, Wd: *i64, Wu: *i64, mWd: *i64, vWd: *i64, mWu: *i64, vWu: *i64, r: i64) -> i64 |
| 135 | func m2_eval_full(tape: *i64, nb: *i64, arena: *i64, ab: *i64, X: *i64, Y: *i64, n: i64, W: *i64) -> i64 |
| 154 | func m2_eval_low(tape: *i64, nb: *i64, arena: *i64, ab: *i64, X: *i64, Y: *i64, n: i64, Wd: *i64, Wu: *i64, r: i64) -> i64 |
| 177 | func m2_sweep_rank(tape: *i64, nb: *i64, arena: *i64, ab: *i64, st: *i64, r: i64, out: *i64) -> i64 |
| 228 | func main() -> i64 |