code wiki / _hdl_build / nx_nofloat_k3interleave.nx
nx_nofloat_k3interleave.nx
buildroot/runtime/_hdl_build/nx_nofloat_k3interleave.nx
about
nx_nofloat_k3interleave.nx -- THE K3 BLOCK STRUCTURE, ASSEMBLED (operator 2026-07-19 "logically integrated,
mature, state of the art"). K3 = DeepSeek-V3/Moonlight skeleton with attention layers INTERLEAVED 3:1 --
three Kimi-Delta-Attention (linear, O(1) KV) layers to one Multi-head-Latent-Attention (softmax, LC-latent
KV) layer (arXiv 2510.26692). This organ proves that pattern COMPOSES from the already-proven parts:
- the "3" KDA layers call the REAL nx_nofloat_kda kda_forward (delta-rule overwrite + per-channel gating)
- the "1" MLA layer calls the REAL nx_nofloat_mla nmla_forward_latent (down-proj latent cache -> softmax)
- every layer: attention sublayer + residual, then FFN sublayer + residual, then RMSNorm (depth-stable)
Schedule: layer l is MLA iff (l % 4 == 3), else KDA -> [KDA,KDA,KDA,MLA] repeating = exactly 3:1.
Pure integer Q16, deterministic AT DEPTH (a float interleaved stack drifts + compounds; ours is bit-exact).
Weight-tied across same-type layers (the composition proof, not a trained net). license_tier: ORIGINAL
No hw writes (Rule 26). Reuses k3_matmul/k3_add/k3_relu/ks_rmsnorm (DRY rule-15).
dependencies 4 imports · 1 importers
imports: nx_nofloat_k3stack.nxnx_nofloat_kda.nxnx_nofloat_mla.nxnx_syscalls.nx
imported by: nx_nofloat_k3interleave_gate.nx
structs
| none |
consts
| 17 | const KIL_Q: i64 = 65536 |
| 18 | const KIL_CELL: i64 = 256 // t*d cap per buffer |
functions
| 21 | func k3il_layer_type(l: i64) -> i64 |
| 28 | func k3il_stack(x: *i64, wq: *i64, wk: *i64, wv: *i64, wo: *i64, w1: *i64, w2: *i64, |