code wiki / _hdl_build / nx_nofloat_attn.nx

nx_nofloat_attn.nx

buildroot/runtime/_hdl_build/nx_nofloat_attn.nx

6013 B80 linesdepth 3pulls 3 transitivereach 0 importersview sourcekind tooltopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_attn.nx -- SOVEREIGN NO-FLOAT ATTENTION (the transformer HEART), integer Q16, deterministic. The hard part of attention is softmax (needs exp); this builds a fixed-point exp via 2^x decomposition (|x|*log2e -> integer shift + quadratic 2^frac), then softmax (max-subtract for stability -> exp -> normalize) + attention (weighted sum of V). KEY: even with an APPROXIMATE exp, softmax NORMALIZES (each / sum), so the weights sum to ONE exactly and monotonicity holds -- all integer => bit-exact deterministic (the moat vs float/CUDA attention). T1 weights non-negative + sum ~= ONE. T2 monotonic (highest logit -> highest weight, order preserved). T3 (EXCEED) determinism bit-identical. T4 attention output = convex combo of V, dominated by the max-score key. expect_exit: 0 Sovereign: nx_syscalls.

dependencies 2 imports · 0 importers

nx_syscalls.nx nx_g_puts_lib.nx nx_nofloat_attn.nx

imports: nx_syscalls.nxnx_g_puts_lib.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main g_puts sys_write g_pn sys_mmap sys_write ↻ exp_fx fxmul sys_mmap ↻ softmax sys_mmap ↻ exp_fx ↻ ck g_puts ↻ attention fxmul ↻ sys_openat_append sys_write ↻ sys_close sys_exit

structs

none

consts

15const Q: i64 = 16
16const ONE: i64 = 65536
17const LOG2E: i64 = 94548 // 1.442695 in Q16
18const C1: i64 = 45426 // ln2 in Q16 (2^f linear term)
19const C2: i64 = 15743 // 0.2402 in Q16 (2^f quadratic term)

functions

12func g_pn(v: i64) -> i64 { let b: *u8=sys_mmap(28); var x: i64=v; if x<0{b[0]=45;sys_write(1,b,1);x=0-x} if x==0{b[0]=48;sys_write(1,b,1);return 0} var d: i64=0; var y: i64=x; while y>0{d=d+1;y=y/10} var i: i64=d-1; y=x; while i>=0{b[i]=(48+(y%10)) as u8;y=y/10;i=i-1} sys_write(1,b,d); return 0 }
called by 1: main calls 2: sys_mmapsys_write
13func ck(name: *u8, c: i64) -> i64 { if c==1 { g_puts(" PASS " as *u8) } else { g_puts(" FAIL " as *u8) } g_puts(name); g_puts("\n" as *u8); return c }
called by 1: main calls 1: g_puts
20func fxmul(a: i64, b: i64) -> i64 { return (a*b)>>Q }
called by 2: exp_fxattention
23func exp_fx(x: i64) -> i64
called by 2: softmaxmain calls 1: fxmul
35func softmax(scores: *i64, w: *i64, n: i64) -> i64
called by 1: main calls 2: sys_mmapexp_fx
42func attention(w: *i64, V: *i64, n: i64) -> i64 { var out: i64=0; var i: i64=0; while i<n { out=out+fxmul(w[i], V[i]); i=i+1 } return out }
called by 1: main calls 1: fxmul
44func main() -> i64