code wiki / _hdl_build / nx_nofloat_k3interleave.nx

nx_nofloat_k3interleave.nx

buildroot/runtime/_hdl_build/nx_nofloat_k3interleave.nx

4356 B90 linesdepth 9pulls 21 transitivereach 1 importersview sourcekind librarytopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_k3interleave.nx -- THE K3 BLOCK STRUCTURE, ASSEMBLED (operator 2026-07-19 "logically integrated, mature, state of the art"). K3 = DeepSeek-V3/Moonlight skeleton with attention layers INTERLEAVED 3:1 -- three Kimi-Delta-Attention (linear, O(1) KV) layers to one Multi-head-Latent-Attention (softmax, LC-latent KV) layer (arXiv 2510.26692). This organ proves that pattern COMPOSES from the already-proven parts: - the "3" KDA layers call the REAL nx_nofloat_kda kda_forward (delta-rule overwrite + per-channel gating) - the "1" MLA layer calls the REAL nx_nofloat_mla nmla_forward_latent (down-proj latent cache -> softmax) - every layer: attention sublayer + residual, then FFN sublayer + residual, then RMSNorm (depth-stable) Schedule: layer l is MLA iff (l % 4 == 3), else KDA -> [KDA,KDA,KDA,MLA] repeating = exactly 3:1. Pure integer Q16, deterministic AT DEPTH (a float interleaved stack drifts + compounds; ours is bit-exact). Weight-tied across same-type layers (the composition proof, not a trained net). license_tier: ORIGINAL No hw writes (Rule 26). Reuses k3_matmul/k3_add/k3_relu/ks_rmsnorm (DRY rule-15).

dependencies 4 imports · 1 importers

nx_nofloat_k3stack.nx nx_nofloat_kda.nx nx_nofloat_mla.nx nx_syscalls.nx nx_nofloat_k3interleave.nx nx_nofloat_k3interleave_gate.nx

imports: nx_nofloat_k3stack.nxnx_nofloat_kda.nxnx_nofloat_mla.nxnx_syscalls.nx

imported by: nx_nofloat_k3interleave_gate.nx

structs

none

consts

17const KIL_Q: i64 = 65536
18const KIL_CELL: i64 = 256 // t*d cap per buffer

functions

21func k3il_layer_type(l: i64) -> i64
called by 2: k3il_stackmain
28func k3il_stack(x: *i64, wq: *i64, wk: *i64, wv: *i64, wo: *i64, w1: *i64, w2: *i64,