code wiki / _hdl_build / nx_nofloat_kda.nx
nx_nofloat_kda.nx
buildroot/runtime/_hdl_build/nx_nofloat_kda.nx
about
nx_nofloat_kda.nx -- KIMI DELTA ATTENTION (KDA) in sovereign no-float = the K3 namesake attention
(operator 2026-07-19 "state of the art... clearly researched, novel, evidence driven"). Extends the proven
nx_nofloat_linattn recurrent kernel with the TWO mechanisms that make KDA distinct (arXiv 2510.26692, Kimi
Linear; 3:1 KDA:MLA in K3):
(a) DELTA RULE (error-correcting write): instead of superposing, KDA overwrites the memory along k --
pred_t = readout(S_{t-1}, k_t) = k_t^T S_{t-1} ; delta_t = beta_t (v_t - pred_t)
S_t = S_{t-1} + k_t (x) delta_t (writing the SAME key twice OVERWRITES, not doubles)
(b) FINE-GRAINED (per-channel diagonal) GATING: each key-channel forgets at its OWN learned rate --
S_t[i][j] = alpha_i * S_{t-1}[i][j] (alpha per key-dim i; GLA has ONE scalar, KDA has d gates)
Combined step: S_t = Diag(alpha) S_{t-1} then delta-write(k_t, v_t, beta_t) ; o_t = q_t S_t.
Pure integer Q16 -> BIT-EXACT deterministic (a float delta/gated stack drifts by accumulation order; ours
does not). Reuses la_readout/la_accum/la_zero (DRY, rule-15). Scratch allocated ONCE per forward (no
per-step mmap -> no leak, the mkv2fmp4 lesson). license_tier: ORIGINAL No hw writes (Rule 26).
dependencies 2 imports · 2 importers
imports: nx_nofloat_linattn.nxnx_syscalls.nx
imported by: nx_nofloat_k3interleave.nxnx_nofloat_kda_gate.nx
structs
| none |
consts
| 17 | const KDA_QBITS: i64 = 16 |
| 18 | const KDA_Q: i64 = 65536 |
functions
| 21 | func kda_gate(s: *i64, alpha: *i64, d: i64) -> i64 |
| 33 | func kda_write(s: *i64, k: *i64, v: *i64, beta: i64, d: i64, pred: *i64, delta: *i64) -> i64 |
| 42 | func kda_forward(qm: *i64, km: *i64, vm: *i64, om: *i64, alpha: *i64, beta: i64, t: i64, d: i64) -> i64 |
| 57 | func kda_step(s: *i64, k: *i64, v: *i64, q: *i64, o: *i64, alpha: *i64, beta: i64, d: i64, pred: *i64, delta: *i64) -> i64 |