code wiki / _hdl_build / nx_nofloat_muon.nx
nx_nofloat_muon.nx
buildroot/runtime/_hdl_build/nx_nofloat_muon.nx
about
nx_nofloat_muon.nx -- the FULL Per-Head Muon optimizer LOOP in sovereign no-float (K3 F405; operator
2026-07-19 "beyond SOTA, novel, evidence-driven"). Muon = MomentUm Orthogonalized by Newton-schulz
(Keller Jordan / Moonshot). One optimizer step on a weight matrix:
B_t = mu * B_{t-1} + G_t (heavy-ball momentum buffer)
U_t = NewtonSchulz(B_t) (orthogonalize the momentum -- the proven nx_nofloat_muon_ns core)
W_t = W_{t-1} - lr * U_t (apply the orthogonalized update)
PER-HEAD: attention weights are nheads independent hd x hd blocks; each is orthogonalized SEPARATELY
(that is the "Per-Head" in Per-Head Muon -- a per-head spectral step, not one global orthogonalization).
PURE INTEGER Q16 throughout -> BIT-EXACT DETERMINISTIC optimizer: a float Muon drifts (non-associative
gradient/momentum accumulation + float NS); ours is identical every machine, every run = the novel TRAIN
exceed. Composes nx_nofloat_muon_ns (mn_orthogonalize). license_tier: ORIGINAL No hw writes (Rule 26).
dependencies 2 imports · 1 importers
imports: nx_nofloat_muon_ns.nxnx_syscalls.nx
imported by: nx_nofloat_muon_gate.nx
structs
| none |
consts
| 15 | const MU_Q: i64 = 65536 |
functions
| 18 | func mu_momentum(buf: *i64, grad: *i64, mu: i64, nn: i64) -> i64 |
| 24 | func mu_apply(w: *i64, u: *i64, lr: i64, nn: i64) -> i64 called by 1: mu_step |
| 30 | func mu_step(w: *i64, buf: *i64, grad: *i64, mu: i64, lr: i64, n: i64, u: *i64) -> i64 |
| 38 | func mu_perhead_step(w: *i64, buf: *i64, grad: *i64, mu: i64, lr: i64, nheads: i64, hd: i64, u: *i64) -> i64 |