code wiki / _hdl_build / nx_nofloat_incontext_gate.nx

nx_nofloat_incontext_gate.nx

buildroot/runtime/_hdl_build/nx_nofloat_incontext_gate.nx

7687 B136 linesdepth 3pulls 4 transitivereach 0 importersview sourcekind gate/prooftopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_incontext_gate.nx -- IN-CONTEXT COPYING (CAP-NF-INCONTEXT): the basis of in-context learning. Task = ECHO: at each position predict the PREVIOUS token (tgt[i]=S[i-1]). EVERY training sequence is FRESH RANDOM (LCG), so the model CANNOT memorize sequences -- it must learn the copy OPERATION (a "previous-token attention head": attend to relative position -1 and copy its token). Then it echoes NEVER-SEEN random sequences -> generalization to unseen DATA, the foundation of in-context learning. T1 held-out echo accuracy >> chance (1/V) on fresh random sequences = the model copies arbitrary unseen content. T2 same on a DIFFERENT test seed = the generalization is robust (not seed-luck). Pure integer Q16, attention-only block (2 clm_fwd call sites). HONEST: copy-back-1 (a previous-token head), the simplest in-context copy -- not full content-based induction. Sovereign: nx_nofloat_autograd + nx_syscalls. expect_exit: 0

dependencies 3 imports · 0 importers

nx_nofloat_autograd.nx nx_syscalls.nx nx_gate_emit_lib.nx nx_nofloat_incontext_gate.nx

imports: nx_nofloat_autograd.nxnx_syscalls.nxnx_gate_emit_lib.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main g_puts sys_write sys_mmap dini do_train make_ex lcg clm_fwd nfa_leaf nfa_new nfa_embed nfa_new ↻ nfa_rmsnorm_rows nfa_new ↻ nfa_qmul nfa_isqrt nfa_matmul nfa_new ↻ nfa_rope nfa_new ↻ nfa_fxexp nfa_cosf nfa_reduce2pi nfa_cosq nfa_qmul ↻ nfa_sinf nfa_reduce2pi ↻ nfa_sinq nfa_qmul ↻ nfa_qmul ↻ nfa_matmul_nt nfa_new ↻ nfa_cmul nfa_new ↻ nfa_qmul ↻ nfa_softmax_rows nfa_new ↻ nfa_fxexp ↻ nfa_vadd

structs

none

consts

14const Q16: i64 = 65536

functions

17func dini(a: *i64, n: i64, sd: i64) -> i64 { var i: i64=0; while i<n { a[i]=(((i*7+sd*13+1)%11)-5)*13107; i=i+1 } return 0 }
called by 1: main
18func lcg(st: *i64) -> i64 { st[0]=(st[0]*1103515245 + 12345) & 2147483647; return (st[0] >> 15) }
called by 1: make_ex
20func make_ex(S: *i64, tgt: *i64, P: i64, V: i64, st: *i64) -> i64
called by 2: do_traineval_echo calls 1: lcg
26func clm_fwd(tape: *i64, vals: *i64, st: *i64, W: *i64, ids: *i64, tgt: *i64, T: i64, dm: i64, V: i64, scale: i64, leaves: *i64) -> i64
54func step_all(tape: *i64, grads: *i64, W: *i64, WN: *i64, leaves: *i64, nW: i64, lr: i64, clip: i64, gb: *i64) -> i64
called by 1: do_train calls 2: nfa_gradnfa_sgd
60func do_train(tape: *i64, vals: *i64, grads: *i64, st: *i64, W: *i64, WN: *i64, S: *i64, tgt: *i64, P: i64, dm: i64, V: i64, scale: i64, leaves: *i64, gb: *i64, steps: i64, sdat: *i64) -> i64
72func eval_echo(tape: *i64, vals: *i64, st: *i64, W: *i64, S: *i64, tgt: *i64, P: i64, dm: i64, V: i64, scale: i64, leaves: *i64, N: i64, sdat: *i64, totp: *i64) -> i64
called by 1: main calls 2: make_exclm_fwd
92func main() -> i64