nx_attention.nx

nx_attention.nx -- scaled dot-product attention kernel. THE kernel that activates everything we've shipped on the sovereign-ML stack. Composes: nx_tensor -- typed shape contract nx_blas_i64 -- gemm-shape Q @ K^T nx_kv_cache -- O(n) autoregressive incremental nx_sparse_tensor -- skip zero entries post-softmax nx_quant_block -- 4-bit Q / K / V at storage layer (queued) nx_numeric_oracle -- bit-exact vs reference verification Per Vaswani 2017: Attention(Q, K, V) = softmax(Q @ K^T / sqrt(d_k)) @ V On the i64 + Q10 substrate: * Q, K, V are i64 tensors with shape [n_tokens, head_dim] * Scores = Q @ K^T (raw integer dot products) * Scale = 1 / sqrt(d_k) approximated via Q10 lookup or constant the caller pre-computes * Softmax in Q10: per-row, find max, subtract, exp via piecewise approximation, normalise * Output = weighted @ V For v1 we expose: nx_attn_score_matrix Q @ K^T raw (i64) nx_attn_scale_q10 apply Q10 scaling nx_attn_softmax_row_q10 per-row softmax in Q10 nx_attn_apply_to_v scores @ V nx_attn_forward full pass (compose the four above) Per the min-hardware-floor cardinal: every step is a separate primitive so the caller can swap in sparse / quant / fused variants at any layer. Softmax-in-Q10 honest accounting: * exp(x) in Q10 uses a 16-entry lookup + linear interp; max error ~3% relative for x in [-3, 0]. * Output rows sum to ~1024 (Q10 unity); rounding pads by up