nx_transformer_block.nx

nx_transformer_block.nx -- full Llama-class transformer block. Composes every per-layer primitive shipped this session into one callable that processes a token sequence through ONE transformer layer. The substrate's load-bearing answer to "off CUDA + run a model" -- callers invoke this function n_layers times and the substrate handles the orchestration. Per the bits-up + no-skipping cardinals: pure composition. No new math, no reinvented kernels. Just wires shipped primitives. ===== Block shape (Llama 2/3 / Mistral / Qwen / Z-Image style) ==== input x: [n_tokens, hidden_dim] 1) attn_in = RMSNorm(x, gamma_attn) 2) q = matmul(attn_in, W_q) : [n_tokens, head_dim*n_heads] k = matmul(attn_in, W_k) : [n_tokens, head_dim*n_kv_heads] v = matmul(attn_in, W_v) : [n_tokens, head_dim*n_kv_heads] 3) q = RoPE(q, positions) : in-place k = RoPE(k, positions) : in-place 4) attn_out = attention(q, k, v) : [n_tokens, hidden_dim] (single-head shape for v1; multi-head + GQA queued) 5) attn_proj = matmul(attn_out, W_o) 6) x = x + attn_proj (residual) 7) ffn_in = RMSNorm(x, gamma_ffn) 8) gate = SiLU(matmul(ffn_in, W_gate)) 9) up = matmul(ffn_in, W_up) 10) ffn_hidden = gate * up (element-wise: SwiGLU) 11) ffn_proj = matmul(ffn_hidden, W_down) 12) x = x + ffn_proj (residual) output: x (same shape as input) Weights packed in a TransformerBlockWeights struct so the caller passes one pointer per block instead of seven matrices. ===== Single-head v1 ============================================ v1 treats hidden_dim as a single attention head. Multi-head and