code wiki / _hdl_build / nx_nofloat_weighttie_gate.nx

nx_nofloat_weighttie_gate.nx

buildroot/runtime/_hdl_build/nx_nofloat_weighttie_gate.nx

7455 B118 linesdepth 3pulls 7 transitivereach 0 importersview sourcekind gate/prooftopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_weighttie_gate.nx -- CAP-NF-WEIGHTTIE: tied input-embedding / output-projection (GPT-style parameter efficiency), pure no-float Q16. The SAME embedding tensor E is used both to embed tokens AND as the output projection (logits = hn . E^T via matmul_nt). One leaf node nE is referenced by two ops, so the autograd tape ACCUMULATES gradients from the embed path + the output path into E -- elegant and exact. Removes the separate Wlm (saves dm*V params). Trained on the 3-category grammar. T1 the tied LM LEARNS (held-out CE << uniform ln(8)=2079). T2 the tied LM is NEAR-OPTIMAL (CE ~= floor 964) -- tying does NOT hurt quality, at fewer params. T3 TEETH: an UNTRAINED tied model has ~uniform CE -> the low CE is from training, not the architecture. Sovereign: nx_nofloat_autograd + nx_syscalls. expect_exit: 0

dependencies 4 imports · 0 importers

nx_nofloat_autograd.nx nx_syscalls.nx nx_gate_emit_lib.nx nx_gate_verdict.nx nx_nofloat_weighttie_gate.nx

imports: nx_nofloat_autograd.nxnx_syscalls.nxnx_gate_emit_lib.nxnx_gate_verdict.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main g_puts sys_write sys_mmap nxa_die sys_write ↻ sys_exit nxa_lock_take nxa_lock_addr sys_write ↻ nxa_lock_give nxa_lock_addr ↻ nxa_report_overrun sys_write ↻ nxa_dump_printable sys_write ↻ nxa_dump_sizes sys_write ↻ dini eval_ce make_stream lcg clm_tied nfa_leaf nfa_new nfa_embed nfa_new ↻ nfa_rmsnorm_rows nfa_new ↻ nfa_qmul nfa_isqrt vm_isqrt nfa_matmul nfa_new ↻ nfa_rope nfa_new ↻ nfa_fxexp nfa_cosf nfa_reduce2pi nfa_cosq

structs

none

consts

14const Q16: i64 = 65536
15const UNIFORM_MNAT: i64 = 2079 // ln(8)
16const FLOOR_MNAT: i64 = 964

functions

19func dini(a: *i64, n: i64, sd: i64) -> i64 { var i: i64=0; while i<n { a[i]=(((i*7+sd*13+1)%11)-5)*13107; i=i+1 } return 0 }
called by 1: main
20func lcg(st: *i64) -> i64 { st[0]=(st[0]*1103515245 + 12345) & 2147483647; return (st[0] >> 15) }
called by 1: make_stream
21func make_stream(S: *i64, tgt: *i64, P: i64, st: *i64) -> i64
called by 2: do_traineval_ce calls 1: lcg
28func clm_tied(tape: *i64, vals: *i64, st: *i64, W: *i64, ids: *i64, tgt: *i64, T: i64, dm: i64, V: i64, scale: i64, lv: *i64) -> i64
54func step_all(tape: *i64, grads: *i64, W: *i64, WN: *i64, lv: *i64, lr: i64, clip: i64, gb: *i64) -> i64
called by 1: do_train calls 2: nfa_gradnfa_sgd
59func do_train(tape: *i64, vals: *i64, grads: *i64, st: *i64, W: *i64, WN: *i64, S: *i64, tgt: *i64, P: i64, dm: i64, V: i64, scale: i64, lv: *i64, gb: *i64, steps: i64, sdat: *i64) -> i64
64func eval_ce(tape: *i64, vals: *i64, st: *i64, W: *i64, S: *i64, tgt: *i64, P: i64, dm: i64, V: i64, scale: i64, lv: *i64, N: i64, sdat: *i64) -> i64
called by 1: main calls 3: make_streamclm_tiednfa_val
71func main() -> i64