code wiki / _hdl_build / nx_fnet_mlm_gate.nx

nx_fnet_mlm_gate.nx

buildroot/runtime/_hdl_build/nx_fnet_mlm_gate.nx

12959 B278 linesdepth 6pulls 16 transitivereach 0 importersview sourcekind gate/prooftopic fnet
docsdependenciesstructsconstsfunctions

about

nx_fnet_mlm_gate.nx -- GATE for MODEL-002: a tiny MASKED LANGUAGE MODEL on the FNet mixer. FNet's Fourier mix is BIDIRECTIONAL (every position sees every other), so its native LM objective is masked-LM (BERT-style), NOT causal next-token (which would let the model see the answer). Here: one position of a 4-token palindrome [a,b,b,a] is replaced by a MASK token; the model fills it in -- which requires MIXING the mirror position into the masked one (a pure bag-of-words model cannot). No attention anywhere. tokens(+MASK) -> EMBED(trained) -> FNET mix -> relu FFN -> softmax-CE over the vocab at the masked slot G_train masked-token accuracy >= 7/8 AND final loss < first loss. G_repro bit-exact: train twice -> identical accuracy + final-loss bits. Evidence -> knowledge/status/fnet_mlm.log (FNETMLMGATE authored=organ ... verdict=GREEN). license_tier: ORIGINAL

dependencies 3 imports · 0 importers

nx_autograd_tensor.nx nx_syscalls.nx nx_gate_verdict.nx nx_fnet_mlm_gate.nx

imports: nx_autograd_tensor.nxnx_syscalls.nxnx_gate_verdict.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap nxa_die sys_write sys_exit nxa_lock_take nxa_lock_addr sys_write ↻ nxa_lock_give nxa_lock_addr ↻ nxa_report_overrun sys_write ↻ nxa_dump_printable sys_write ↻ nxa_dump_sizes sys_write ↻ ml_train ta_det_init ta_constf nx_f32_div nx_f32_classify nx_f32_exp_field nx_f32_mant_field nx_f32_sign nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_i32_to_f32 sys_mmap ↻ ta_constf ↻ ml_build ta_leaf ta_new sys_mmap ↻ ml_embed ta_fnet ta_new ↻ sys_mmap ↻ fnet_mix_f32 sys_mmap ↻ fftf_twiddles

structs

none

consts

16const MV: i64 = 5 // input vocab: tokens 0..3 + MASK=4
17const MN: i64 = 4 // sequence length
18const MD: i64 = 4 // d_model
19const MH: i64 = 8 // FFN hidden
20const MC: i64 = 4 // output vocab (predict token 0..3)
21const MND: i64 = 16 // MN*MD
23const ML_LOG: *u8 = "knowledge/status/fnet_mlm.log"

functions

25func ml_w(fd: i64, s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(fd, s, n); return 0 }
called by 1: main calls 1: sys_write
26func ml_wn(fd: i64, v: i64) -> i64
called by 1: main calls 2: sys_mmapsys_write
37func ml_embed(E: *i64, seq: *i64, soff: i64, xout: *i64) -> i64
called by 2: ml_buildml_predict
48func ml_fwd(tape: *i64, vals: *i64, st: *i64, x: *i64, nW1: i64, nb1: i64, nW2: i64, nb2: i64) -> i64
55func ml_build(tape: *i64, vals: *i64, st: *i64, E: *i64, W1: *i64, b1: *i64, W2: *i64, b2: *i64,
86func ml_predict(tape: *i64, vals: *i64, st: *i64, E: *i64, W1: *i64, b1: *i64, W2: *i64, b2: *i64, seqs: *i64, s: i64) -> i64
106func ml_adamw(p: *i64, m: *i64, v: *i64, g: *i64, n: i64, lr: i64, beta1: i64, beta2: i64, om1: i64, om2: i64, eps: i64, c1: i64, c2: i64) -> i64
118func ml_train(tape: *i64, vals: *i64, grads: *i64, st: *i64, E: *i64, W1: *i64, b1: *i64, W2: *i64, b2: *i64,
196func main() -> i64