code wiki / _hdl_build / nx_nofloat_k3stack.nx
nx_nofloat_k3stack.nx
buildroot/runtime/_hdl_build/nx_nofloat_k3stack.nx
about
nx_nofloat_k3stack.nx -- MULTI-LAYER K3 stack with ATTENTION RESIDUALS in no-float (operator 2026-07-19
"logically integrated, mature"). Stacks the K3 block at DEPTH, and closes the K3-specific Attention-
Residuals gap: instead of a plain x+f(x) residual, each layer's input is a CONTENT-WEIGHTED ATTENTION
over ALL preceding layer outputs (arXiv 2603.15031: "learned input-dependent attention over preceding
layers... earlier-layer representations stay reachable deeper in the stack"). Minimal faithful AttnRes:
score_s = |mean(h_s) . mean(h_{l-1})| ; w_s = score_s / sum ; r_l = sum_{s<l} w_s h_s ; h_l = block(r_l)
Pure integer Q16, deterministic AT DEPTH -- a float stack drifts per-layer and compounds; ours is bit-exact
through N layers. Composes nx_nofloat_k3block (the K3 block) + weight-tying across layers.
license_tier: ORIGINAL No hw writes (Rule 26).
dependencies 2 imports · 2 importers
imports: nx_nofloat_k3block.nxnx_syscalls.nx
imported by: nx_nofloat_k3interleave.nxnx_nofloat_k3stack_gate.nx
structs
| none |
consts
| 13 | const KS_QBITS: i64 = 16 |
| 14 | const KS_Q: i64 = 65536 |
| 15 | const KS_LMAX: i64 = 8 |
| 16 | const KS_CELLMAX: i64 = 256 // t*d cap per layer buffer |
functions
| 18 | func ks_mean(h: *i64, td: i64) -> i64 called by 1: k3_stack |
| 24 | func ks_isqrt(v: i64) -> i64 called by 1: ks_rmsnorm |
| 33 | func ks_rmsnorm(h: *i64, td: i64) -> i64 |
| 44 | func k3_stack(x: *i64, wq: *i64, wk: *i64, wv: *i64, wo: *i64, w1: *i64, w2: *i64, layers: i64, t: i64, d: i64, dff: i64, out: *i64) -> i64 |
| 95 | func k3_stack_out_layer(x: *i64, wq: *i64, wk: *i64, wv: *i64, wo: *i64, w1: *i64, w2: *i64, layers: i64, t: i64, d: i64, dff: i64, out: *i64) -> i64 calls 1: k3_stack |