code wiki / (root) / nx_f32_attention.nx

nx_f32_attention.nx

buildroot/runtime/nx_f32_attention.nx

4763 B104 linesdepth 6pulls 8 transitivereach 7 importersview sourcekind tooltopic f32
docsdependenciesstructsconstsfunctions

about

nx_f32_attention.nx -- software-f32 multi-token scaled-dot-product attention (single head), the first brick of the DiT diffusion-backbone half of the sd-server -> Nishi migration. `nx_f32_mha.nx` is SINGLE-TOKEN (autoregressive decode, n_tokens=1, degenerate softmax). The DiT needs MULTI-TOKEN / bidirectional self-attention over all latent (+text) tokens -- exactly the follow-on its header flags ("adds K^T transpose + multi-row softmax"). This is the standard prefill attention: scores[i][j] = dot(Q[i], K[j]) * scale (Q@K^T, scaled) probs[i] = softmax(scores[i]) (over the j axis, numerically stable) out[i] = sum_j probs[i][j] * V[j] (probs@V) Composes the gated primitives nx_f32_add/mul + nx_f32_softmax. Single-head; multi-head = caller loops heads over the head_dim slices of Q/K/V (RoPE + QK-RMSNorm wrap this in the DiT block, next rung). Q,K,V,out: flat *i64 f32 bits, [n_tokens, head_dim] row-major. score_row/prob_row: caller scratch [n_tokens]. license_tier: ORIGINAL

dependencies 5 imports · 3 importers

nx_syscalls.nx nx_f32.nx nx_f32_div.nx nx_f32_cvt.nx nx_f32_softmax.nx nx_f32_attention.nx nx_f32_dit_block_tiny.nx nx_f32_gqa_attention.nx nx_linear_attn_bench.nx

imports: nx_syscalls.nxnx_f32.nxnx_f32_div.nxnx_f32_cvt.nxnx_f32_softmax.nx

imported by: nx_f32_dit_block_tiny.nxnx_f32_gqa_attention.nxnx_linear_attn_bench.nx

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap nx_i32_to_f32 nx_f32_div nx_f32_classify nx_f32_exp_field nx_f32_mant_field nx_f32_sign nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_f32_attention nx_f32_add nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_f32_mul nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_f32_softmax _f32_gt nx_f32_sub nx_f32_add ↻ nx_f32_neg nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_sub ↻ nx_f32_exp nx_f32_classify ↻ nx_f32_sign ↻ _f32_to_i32_rne nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_exp_field ↻ nx_f32_mant_field ↻ nx_i32_to_f32 ↻ _f32_ldexp nx_f32_classify ↻

structs

none

consts

22const NX_F32ATTN_OK: i64 = 0
23const NX_F32ATTN_ERR: i64 = 1

functions

25func nx_f32_attention(Q: *i64, K: *i64, V: *i64, n_tokens: i64, head_dim: i64, scale: i64,
67func main() -> i64