code wiki / _hdl_build / nx_nofloat_kda_gate.nx
nx_nofloat_kda_gate.nx
buildroot/runtime/_hdl_build/nx_nofloat_kda_gate.nx
about
nx_nofloat_kda_gate.nx -- gate: Kimi Delta Attention, the K3-distinctive behaviors (not just "it runs").
T1 DELTA OVERWRITE: writing the SAME key twice (v1 then v2) OVERWRITES to v2 -- vs plain linear-attn
which superposes to v1+v2. This is the delta-rule signature (key-value memory, not accumulation).
T2 CHANNEL GATING DECAY: a stored value decays monotonically under repeated per-channel gating (alpha<1).
T3 GATE=1 PERSISTENCE: with alpha=1 and no write, the memory is bit-identical across steps (stable).
T4 DETERMINISTIC: kda_forward twice = bit-identical (the exceed: float delta/gated stacks drift).
T5 FINE-GRAINED (per-channel): two key-channels with DIFFERENT alpha decay at different rates -> the
cross-channel magnitude RATIO changes (scalar/GLA gating would preserve it). The KDA-vs-GLA distinction.
license_tier: ORIGINAL No hw writes (Rule 26). expect_exit: 0
dependencies 4 imports · 0 importers
imports: nx_nofloat_kda.nxnx_nofloat_linattn.nxnx_gate_verdict.nxnx_syscalls.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 15 | const KG_Q: i64 = 65536 |
| 16 | const KG_D: i64 = 4 |
functions
| 18 | func kg_abs(x: i64) -> i64 { if x < 0 { return 0 - x } return x } called by 1: main |
| 20 | func main() -> i64 |