code wiki / _hdl_build / nx_fnet_mlm_gate.nx

nx_fnet_mlm_gate.nx

buildroot/runtime/_hdl_build/nx_fnet_mlm_gate.nx

12425 B270 linesdepth 6pulls 15 transitivereach 0 importersview sourcekind gate/prooftopic fnet
docsdependenciesstructsconstsfunctions

about

nx_fnet_mlm_gate.nx -- GATE for MODEL-002: a tiny MASKED LANGUAGE MODEL on the FNet mixer. FNet's Fourier mix is BIDIRECTIONAL (every position sees every other), so its native LM objective is masked-LM (BERT-style), NOT causal next-token (which would let the model see the answer). Here: one position of a 4-token palindrome [a,b,b,a] is replaced by a MASK token; the model fills it in -- which requires MIXING the mirror position into the masked one (a pure bag-of-words model cannot). No attention anywhere. tokens(+MASK) -> EMBED(trained) -> FNET mix -> relu FFN -> softmax-CE over the vocab at the masked slot G_train masked-token accuracy >= 7/8 AND final loss < first loss. G_repro bit-exact: train twice -> identical accuracy + final-loss bits. Evidence -> knowledge/status/fnet_mlm.log (FNETMLMGATE authored=organ ... verdict=GREEN). license_tier: ORIGINAL

dependencies 2 imports · 0 importers

nx_autograd_tensor.nx nx_syscalls.nx nx_fnet_mlm_gate.nx

imports: nx_autograd_tensor.nxnx_syscalls.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap ml_train ta_det_init ta_constf nx_f32_div nx_f32_classify nx_f32_exp_field nx_f32_mant_field nx_f32_sign nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_i32_to_f32 sys_mmap ↻ ta_constf ↻ ml_build ta_leaf ta_new sys_mmap ↻ ml_embed ta_fnet ta_new ↻ sys_mmap ↻ fnet_mix_f32 sys_mmap ↻ fftf_twiddles fftf_q16_to_f32 fx_cos fx_sin fftf_fwd nx_fft_log2 nx_fft_bit_reverse nx_f32_sub nx_f32_mul nx_f32_add ta_relu ta_new ↻ nx_f32_gt nx_f32_lt nx_f32_is_nan

structs

none

consts

15const MV: i64 = 5 // input vocab: tokens 0..3 + MASK=4
16const MN: i64 = 4 // sequence length
17const MD: i64 = 4 // d_model
18const MH: i64 = 8 // FFN hidden
19const MC: i64 = 4 // output vocab (predict token 0..3)
20const MND: i64 = 16 // MN*MD
22const ML_LOG: *u8 = "knowledge/status/fnet_mlm.log"

functions

24func ml_w(fd: i64, s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(fd, s, n); return 0 }
called by 1: main calls 1: sys_write
25func ml_wn(fd: i64, v: i64) -> i64
called by 1: main calls 2: sys_mmapsys_write
36func ml_embed(E: *i64, seq: *i64, soff: i64, xout: *i64) -> i64
called by 2: ml_buildml_predict
47func ml_fwd(tape: *i64, vals: *i64, st: *i64, x: *i64, nW1: i64, nb1: i64, nW2: i64, nb2: i64) -> i64
54func ml_build(tape: *i64, vals: *i64, st: *i64, E: *i64, W1: *i64, b1: *i64, W2: *i64, b2: *i64,
85func ml_predict(tape: *i64, vals: *i64, st: *i64, E: *i64, W1: *i64, b1: *i64, W2: *i64, b2: *i64, seqs: *i64, s: i64) -> i64
105func ml_adamw(p: *i64, m: *i64, v: *i64, g: *i64, n: i64, lr: i64, beta1: i64, beta2: i64, om1: i64, om2: i64, eps: i64, c1: i64, c2: i64) -> i64
117func ml_train(tape: *i64, vals: *i64, grads: *i64, st: *i64, E: *i64, W1: *i64, b1: *i64, W2: *i64, b2: *i64,
195func main() -> i64