code wiki / _hdl_build / nx_nofloat_kda.nx

nx_nofloat_kda.nx

buildroot/runtime/_hdl_build/nx_nofloat_kda.nx

3482 B62 linesdepth 3pulls 3 transitivereach 3 importersview sourcekind librarytopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_kda.nx -- KIMI DELTA ATTENTION (KDA) in sovereign no-float = the K3 namesake attention (operator 2026-07-19 "state of the art... clearly researched, novel, evidence driven"). Extends the proven nx_nofloat_linattn recurrent kernel with the TWO mechanisms that make KDA distinct (arXiv 2510.26692, Kimi Linear; 3:1 KDA:MLA in K3): (a) DELTA RULE (error-correcting write): instead of superposing, KDA overwrites the memory along k -- pred_t = readout(S_{t-1}, k_t) = k_t^T S_{t-1} ; delta_t = beta_t (v_t - pred_t) S_t = S_{t-1} + k_t (x) delta_t (writing the SAME key twice OVERWRITES, not doubles) (b) FINE-GRAINED (per-channel diagonal) GATING: each key-channel forgets at its OWN learned rate -- S_t[i][j] = alpha_i * S_{t-1}[i][j] (alpha per key-dim i; GLA has ONE scalar, KDA has d gates) Combined step: S_t = Diag(alpha) S_{t-1} then delta-write(k_t, v_t, beta_t) ; o_t = q_t S_t. Pure integer Q16 -> BIT-EXACT deterministic (a float delta/gated stack drifts by accumulation order; ours does not). Reuses la_readout/la_accum/la_zero (DRY, rule-15). Scratch allocated ONCE per forward (no per-step mmap -> no leak, the mkv2fmp4 lesson). license_tier: ORIGINAL No hw writes (Rule 26).

dependencies 2 imports · 2 importers

nx_nofloat_linattn.nx nx_syscalls.nx nx_nofloat_kda.nx nx_nofloat_k3interleave.nx nx_nofloat_kda_gate.nx

imports: nx_nofloat_linattn.nxnx_syscalls.nx

imported by: nx_nofloat_k3interleave.nxnx_nofloat_kda_gate.nx

structs

none

consts

17const KDA_QBITS: i64 = 16
18const KDA_Q: i64 = 65536

functions

21func kda_gate(s: *i64, alpha: *i64, d: i64) -> i64
33func kda_write(s: *i64, k: *i64, v: *i64, beta: i64, d: i64, pred: *i64, delta: *i64) -> i64
42func kda_forward(qm: *i64, km: *i64, vm: *i64, om: *i64, alpha: *i64, beta: i64, t: i64, d: i64) -> i64
57func kda_step(s: *i64, k: *i64, v: *i64, q: *i64, o: *i64, alpha: *i64, beta: i64, d: i64, pred: *i64, delta: *i64) -> i64