nx_f32_attention.nx
buildroot/runtime/nx_f32_attention.nx
about
nx_f32_attention.nx -- software-f32 multi-token scaled-dot-product attention (single head), the first
brick of the DiT diffusion-backbone half of the sd-server -> Nishi migration.
`nx_f32_mha.nx` is SINGLE-TOKEN (autoregressive decode, n_tokens=1, degenerate softmax). The DiT needs
MULTI-TOKEN / bidirectional self-attention over all latent (+text) tokens -- exactly the follow-on its
header flags ("adds K^T transpose + multi-row softmax"). This is the standard prefill attention:
scores[i][j] = dot(Q[i], K[j]) * scale (Q@K^T, scaled)
probs[i] = softmax(scores[i]) (over the j axis, numerically stable)
out[i] = sum_j probs[i][j] * V[j] (probs@V)
Composes the gated primitives nx_f32_add/mul + nx_f32_softmax. Single-head; multi-head = caller loops
heads over the head_dim slices of Q/K/V (RoPE + QK-RMSNorm wrap this in the DiT block, next rung).
Q,K,V,out: flat *i64 f32 bits, [n_tokens, head_dim] row-major. score_row/prob_row: caller scratch [n_tokens].
license_tier: ORIGINAL
dependencies 5 imports · 3 importers
imports: nx_syscalls.nxnx_f32.nxnx_f32_div.nxnx_f32_cvt.nxnx_f32_softmax.nx
imported by: nx_f32_dit_block_tiny.nxnx_f32_gqa_attention.nxnx_linear_attn_bench.nx
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 22 | const NX_F32ATTN_OK: i64 = 0 |
| 23 | const NX_F32ATTN_ERR: i64 = 1 |
functions
| 25 | func nx_f32_attention(Q: *i64, K: *i64, V: *i64, n_tokens: i64, head_dim: i64, scale: i64, called by 4: mainnx_f32_dit_block_tinynx_f32_gqa_attentionmain calls 3: nx_f32_addnx_f32_mulnx_f32_softmax |
| 67 | func main() -> i64 |