code wiki / _hdl_build / nx_nofloat_adamw32_gate.nx

nx_nofloat_adamw32_gate.nx

buildroot/runtime/_hdl_build/nx_nofloat_adamw32_gate.nx

5412 B86 linesdepth 3pulls 7 transitivereach 0 importersview sourcekind gate/prooftopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_adamw32_gate.nx -- the REAL win from the R4 scale-up work: a numerically STABLE fixed-point Adam. DIAGNOSIS: the Q16 AdamW (lib nfa_adamw) DIVERGES/STALLS on small gradients because g^2 UNDERFLOWS in Q16 -- e.g. g=0.06 (Q16 3932) -> qmul(g,g)=235 (0.0036) and the (1-b2)*g^2 increment qmul(66,235)=0, so the 2nd moment v never accumulates, the denominator collapses to eps, and the update mhat/eps explodes (or stalls). FIX: accumulate v in Q32 and compute g^2 = g*g at FULL precision (no >>16) so small gradients survive. T1: Q32-moment AdamW CONVERGES on a small-gradient fit (loss -> ~0). T2 (measured exceed): Q32 final loss << Q16 final loss on the SAME problem (the underflow fix is real). Self-contained controlled experiment (n=8 fit, hand-coded grad 2*(w-t), no transformer). Pure integer. expect_exit: 0 Sovereign: nx_nofloat_autograd (for nfa_adamw + nfa_isqrt) + nx_syscalls.

dependencies 4 imports · 0 importers

nx_nofloat_autograd.nx nx_syscalls.nx nx_gate_emit_lib.nx nx_gate_verdict.nx nx_nofloat_adamw32_gate.nx

imports: nx_nofloat_autograd.nxnx_syscalls.nxnx_gate_emit_lib.nxnx_gate_verdict.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main g_puts sys_write sys_mmap nxa_die sys_write ↻ sys_exit nxa_lock_take nxa_lock_addr sys_write ↻ nxa_lock_give nxa_lock_addr ↻ nxa_report_overrun sys_write ↻ nxa_dump_printable sys_write ↻ nxa_dump_sizes sys_write ↻ fill_grad nfa_adamw nfa_qmul nfa_isqrt vm_isqrt loss_q32 adamw32_one nfa_isqrt ↻ g_pn sys_mmap ↻ sys_write ↻ g_check g_puts ↻ sys_openat_append sys_write ↻ sys_close gv_ctr sys_mmap ↻ gv_verdict gv_note_bare_rate gv_bare_rate gv_at

structs

none

consts

14const Q16: i64 = 65536

functions

19func adamw32_one(w: *i64, g: *i64, m: *i64, v: *i64, n: i64, lr: i64, b1: i64, b2: i64, eps: i64, t: i64) -> i64
called by 1: main calls 1: nfa_isqrt
40func loss_q32(w: *i64, t: *i64, n: i64) -> i64 { var s: i64=0; var i: i64=0; while i<n { let d: i64=w[i]-t[i]; s=s+d*d; i=i+1 } return s }
called by 1: main
42func fill_grad(w: *i64, t: *i64, g: *i64, n: i64) -> i64 { var i: i64=0; while i<n { g[i]=2*(w[i]-t[i]); i=i+1 } return 0 }
called by 1: main
44func main() -> i64