code wiki / _hdl_build / nx_nofloat_adamw32_gate.nx

nx_nofloat_adamw32_gate.nx

buildroot/runtime/_hdl_build/nx_nofloat_adamw32_gate.nx

5029 B78 linesdepth 3pulls 4 transitivereach 0 importersview sourcekind gate/prooftopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_adamw32_gate.nx -- the REAL win from the R4 scale-up work: a numerically STABLE fixed-point Adam. DIAGNOSIS: the Q16 AdamW (lib nfa_adamw) DIVERGES/STALLS on small gradients because g^2 UNDERFLOWS in Q16 -- e.g. g=0.06 (Q16 3932) -> qmul(g,g)=235 (0.0036) and the (1-b2)*g^2 increment qmul(66,235)=0, so the 2nd moment v never accumulates, the denominator collapses to eps, and the update mhat/eps explodes (or stalls). FIX: accumulate v in Q32 and compute g^2 = g*g at FULL precision (no >>16) so small gradients survive. T1: Q32-moment AdamW CONVERGES on a small-gradient fit (loss -> ~0). T2 (measured exceed): Q32 final loss << Q16 final loss on the SAME problem (the underflow fix is real). Self-contained controlled experiment (n=8 fit, hand-coded grad 2*(w-t), no transformer). Pure integer. expect_exit: 0 Sovereign: nx_nofloat_autograd (for nfa_adamw + nfa_isqrt) + nx_syscalls.

dependencies 3 imports · 0 importers

nx_nofloat_autograd.nx nx_syscalls.nx nx_gate_emit_lib.nx nx_nofloat_adamw32_gate.nx

imports: nx_nofloat_autograd.nxnx_syscalls.nxnx_gate_emit_lib.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main g_puts sys_write sys_mmap fill_grad nfa_adamw nfa_qmul nfa_isqrt loss_q32 adamw32_one nfa_isqrt ↻ g_pn sys_mmap ↻ sys_write ↻ g_check g_puts ↻ sys_openat_append sys_write ↻ sys_close sys_exit

structs

none

consts

13const Q16: i64 = 65536

functions

18func adamw32_one(w: *i64, g: *i64, m: *i64, v: *i64, n: i64, lr: i64, b1: i64, b2: i64, eps: i64, t: i64) -> i64
called by 1: main calls 1: nfa_isqrt
39func loss_q32(w: *i64, t: *i64, n: i64) -> i64 { var s: i64=0; var i: i64=0; while i<n { let d: i64=w[i]-t[i]; s=s+d*d; i=i+1 } return s }
called by 1: main
41func fill_grad(w: *i64, t: *i64, g: *i64, n: i64) -> i64 { var i: i64=0; while i<n { g[i]=2*(w[i]-t[i]); i=i+1 } return 0 }
called by 1: main
43func main() -> i64