code wiki / _hdl_build / nx_nofloat_scale_batch_gate.nx
nx_nofloat_scale_batch_gate.nx
buildroot/runtime/_hdl_build/nx_nofloat_scale_batch_gate.nx
about
nx_nofloat_scale_batch_gate.nx -- R4 retry via the ONE untested fundamental lever: MINI-BATCH GRADIENT
AVERAGING. All prior richer-grammar training was BATCH-1 (one fresh-random stream/step = very noisy gradients
-> SGD oscillates, can't sharpen to the floor). Standard fix: average gradients over a mini-batch before
stepping. Same richer 4-cat grammar (DET2->ADJ4->NOUN5->VERB5, vocab 16) + dm=32 that plateaued at CE ~2135
(ppl 8.5); here we accumulate grads over B=16 fresh streams per step (low-noise), then ONE SGD step.
floor = avg(ln2,ln4,ln5,ln5)=1324 milli-nats (ppl 3.76); uniform = ln(16)=2773.
T1 held-out CE << uniform (learned). T2 held-out CE ~= floor (near-OPTIMAL -> batch-averaging cracked it).
If T2 fails too, batch size is NOT the lever and the tractable training levers are genuinely exhausted (honest).
expect_exit: 0 Sovereign: nx_nofloat_autograd + nx_syscalls.
dependencies 3 imports · 0 importers
imports: nx_nofloat_autograd.nxnx_syscalls.nxnx_gate_emit_lib.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 13 | const Q16: i64 = 65536 |
| 14 | const UNIFORM_MNAT: i64 = 2773 |
| 15 | const FLOOR_MNAT: i64 = 1324 |
functions
| 18 | func dini(a: *i64, n: i64, sd: i64) -> i64 { var i: i64=0; while i<n { a[i]=(((i*7+sd*13+1)%11)-5)*13107; i=i+1 } return 0 } called by 1: main |
| 19 | func lcg(st: *i64) -> i64 { st[0]=(st[0]*1103515245 + 12345) & 2147483647; return (st[0] >> 15) } called by 1: make_stream4 |
| 20 | func make_stream4(S: *i64, tgt: *i64, P: i64, st: *i64) -> i64 |
| 26 | func clm_fwd(tape: *i64, vals: *i64, st: *i64, W: *i64, ids: *i64, tgt: *i64, T: i64, dm: i64, V: i64, scale: i64, leaves: *i64) -> i64 called by 2: do_train_batcheval_ce calls 10: nfa_leafnfa_embednfa_rmsnorm_rowsnfa_matmulnfa_ropenfa_matmul_nt+4 |
| 56 | func do_train_batch(tape: *i64, vals: *i64, grads: *i64, st: *i64, W: *i64, WN: *i64, gacc: *i64, S: *i64, tgt: *i64, P: i64, dm: i64, V: i64, scale: i64, leaves: *i64, gb: *i64, outer: i64, B: i64, lr: i64, sdat: *i64) -> i64 |
| 76 | func eval_ce(tape: *i64, vals: *i64, st: *i64, W: *i64, S: *i64, tgt: *i64, P: i64, dm: i64, V: i64, scale: i64, leaves: *i64, N: i64, sdat: *i64) -> i64 |
| 83 | func main() -> i64 |