code wiki / _hdl_build / nx_nofloat_incontext_gate.nx
nx_nofloat_incontext_gate.nx
buildroot/runtime/_hdl_build/nx_nofloat_incontext_gate.nx
about
nx_nofloat_incontext_gate.nx -- IN-CONTEXT COPYING (CAP-NF-INCONTEXT): the basis of in-context learning.
Task = ECHO: at each position predict the PREVIOUS token (tgt[i]=S[i-1]). EVERY training sequence is FRESH
RANDOM (LCG), so the model CANNOT memorize sequences -- it must learn the copy OPERATION (a "previous-token
attention head": attend to relative position -1 and copy its token). Then it echoes NEVER-SEEN random
sequences -> generalization to unseen DATA, the foundation of in-context learning.
T1 held-out echo accuracy >> chance (1/V) on fresh random sequences = the model copies arbitrary unseen content.
T2 same on a DIFFERENT test seed = the generalization is robust (not seed-luck).
Pure integer Q16, attention-only block (2 clm_fwd call sites). HONEST: copy-back-1 (a previous-token head),
the simplest in-context copy -- not full content-based induction. Sovereign: nx_nofloat_autograd + nx_syscalls.
expect_exit: 0
dependencies 3 imports · 0 importers
imports: nx_nofloat_autograd.nxnx_syscalls.nxnx_gate_emit_lib.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 14 | const Q16: i64 = 65536 |
functions
| 17 | func dini(a: *i64, n: i64, sd: i64) -> i64 { var i: i64=0; while i<n { a[i]=(((i*7+sd*13+1)%11)-5)*13107; i=i+1 } return 0 } called by 1: main |
| 18 | func lcg(st: *i64) -> i64 { st[0]=(st[0]*1103515245 + 12345) & 2147483647; return (st[0] >> 15) } called by 1: make_ex |
| 20 | func make_ex(S: *i64, tgt: *i64, P: i64, V: i64, st: *i64) -> i64 |
| 26 | func clm_fwd(tape: *i64, vals: *i64, st: *i64, W: *i64, ids: *i64, tgt: *i64, T: i64, dm: i64, V: i64, scale: i64, leaves: *i64) -> i64 called by 2: do_traineval_echo calls 10: nfa_leafnfa_embednfa_rmsnorm_rowsnfa_matmulnfa_ropenfa_matmul_nt+4 |
| 54 | func step_all(tape: *i64, grads: *i64, W: *i64, WN: *i64, leaves: *i64, nW: i64, lr: i64, clip: i64, gb: *i64) -> i64 |
| 60 | func do_train(tape: *i64, vals: *i64, grads: *i64, st: *i64, W: *i64, WN: *i64, S: *i64, tgt: *i64, P: i64, dm: i64, V: i64, scale: i64, leaves: *i64, gb: *i64, steps: i64, sdat: *i64) -> i64 |
| 72 | func eval_echo(tape: *i64, vals: *i64, st: *i64, W: *i64, S: *i64, tgt: *i64, P: i64, dm: i64, V: i64, scale: i64, leaves: *i64, N: i64, sdat: *i64, totp: *i64) -> i64 |
| 92 | func main() -> i64 |