code wiki / _hdl_build / nx_nofloat_xformer_gate.nx
nx_nofloat_xformer_gate.nx
buildroot/runtime/_hdl_build/nx_nofloat_xformer_gate.nx
about
nx_nofloat_xformer_gate.nx -- HARD-EVIDENCE gate for the TRANSFORMER-SUBLAYER backward ops added to the
no-float autograd (softmax / silu / rmsnorm): the nonlinearities of a Qwen block, now DIFFERENTIABLE in
pure integer Q16. This is the CAP-NF-TRAIN-XFORMER rung: backprop through the transformer's own ops.
A GRADCHECK each op (oracle = mathematics): build loss = mse(OP(x), target), compare the tape's analytic
reverse-sweep grad dL/dx_i against a CENTRAL finite difference (L(x+h)-L(x-h))/2h, all Q16. Done for
softmax (Jacobian-vector product), silu (x*sigmoid(x) derivative), rmsnorm (normalization Jacobian).
D TEETH (neg-control): negate one analytic grad and assert the SAME check now FAILS -> a wrong gradient
is provably caught (the agreement in A is real, not vacuous).
B A TRANSFORMER ACTIVATION TRAINS: fit y = silu(w*.x) (realizable, w*=1.5) from w=0 by GD through the
tape+silu backward. Assert loss collapses >=90% and w converges to w*. Proves the FFN nonlinearity is
trainable end-to-end in pure integer.
C BIT-EXACT: train twice from zero -> identical integer w (determinism is structural for integer).
Evidence -> knowledge/status/nofloat_xformer.log. Sovereign: imports nx_nofloat_autograd (pure integer; NO
nx_f32 in the import graph) + nx_syscalls. license_tier: ORIGINAL expect_exit: 0
dependencies 5 imports · 0 importers
imports: nx_nofloat_autograd.nxnx_g_check_lib.nxnx_g_pn_lib.nxnx_g_puts_lib.nxnx_syscalls.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 23 | const XLOG: *u8 = "knowledge/status/nofloat_xformer.log" |
| 24 | const Q16: i64 = 65536 |
| 25 | const OP_SOFTMAX: i64 = 5 |
| 26 | const OP_SILU: i64 = 6 |
| 27 | const OP_RMSNORM: i64 = 7 |
functions
| 29 | func g_abs(v: i64) -> i64 { if v < 0 { return 0 - v } return v } |
| 30 | func q_milli(q: i64) -> i64 { var neg: i64=0; var a: i64=q; if a<0 { neg=1; a=0-a } let m: i64=(a*1000)/Q16; if neg==1 { return 0-m } return m } called by 1: main |
| 31 | func x_ws(fd: i64, s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(fd,s,n); return 0 } |
| 32 | func x_wn(fd: i64, v: i64) -> i64 |
| 40 | func xf_loss(tape: *i64, vals: *i64, st: *i64, op_sel: i64, xs: *i64, ts: *i64, n: i64, leaves: *i64) -> i64 |
| 52 | func xf_lossval(tape: *i64, vals: *i64, st: *i64, op_sel: i64, xs: *i64, ts: *i64, n: i64) -> i64 |
| 58 | func xf_fd(tape: *i64, vals: *i64, st: *i64, op_sel: i64, xs: *i64, ts: *i64, n: i64, pi: i64, h: i64) -> i64 |
| 69 | func xf_gradcheck(tape: *i64, vals: *i64, grads: *i64, st: *i64, op_sel: i64, xs: *i64, ts: *i64, n: i64, h: i64, tol_q: i64, floor_q: i64, worst: *i64) -> i64 |
| 91 | func xf_silu_build(tape: *i64, vals: *i64, st: *i64, wp: *i64, X: *i64, Y: *i64, T: i64, outW: *i64) -> i64 |
| 108 | func xf_silu_train(tape: *i64, vals: *i64, grads: *i64, st: *i64, epochs: i64, lr_q: i64, wout: *i64, lf: *i64, ll: *i64) -> i64 |
| 133 | func main() -> i64 |