nx_f32_qwen_layer.nx
buildroot/runtime/nx_f32_qwen_layer.nx
about
nx_f32_qwen_layer.nx -- a FULL sovereign f32 Qwen transformer layer (the Z-Image text-encoder + companion
chat model's repeating unit), assembled from the gated bricks.
sd-server -> Nishi migration: the pre-norm GQA transformer layer (Llama/Qwen family). Per the real GGUF:
RMSNorm -> [Wq/Wk/Wv] -> GQA attention (32 q / 8 kv heads) -> Wo -> residual -> RMSNorm -> SwiGLU
(ffn_gate/ffn_up -> SiLU-gate -> ffn_down) -> residual. Composes nx_f32_rmsnorm + nx_f32_gqa_attention +
nx_f32_silu + inline linears. Stack this x n_layers (a loop) = the full encoder forward -> real prompt
embeddings. Same layer serves image-prompt-encoding AND companion chat (the shared-LLM reuse).
x,out: flat *i64 f32 bits [n_tokens, hidden]. W{q}: [q_dim,hidden], W{k,v}: [kv_dim,hidden], Wo: [hidden,q_dim];
ffn Wg/Wu: [ffn_dim,hidden], Wd: [hidden,ffn_dim]; an_g/fn_g: [hidden]. Scratch is internal (fork-per-use).
license_tier: ORIGINAL
dependencies 7 imports · 1 importers
imports: nx_syscalls.nxnx_f32.nxnx_f32_div.nxnx_f32_cvt.nxnx_f32_rmsnorm.nxnx_f32_gqa_attention.nxnx_f32_activations.nx
imported by: nx_f32_qwen_encoder.nx
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 20 | const K_MAGIC_1000000: i64 = 1000000 |
functions
| 22 | func qw_linear(inp: *i64, n_tokens: i64, in_dim: i64, W: *i64, out_dim: i64, out: *i64) -> i64 |
| 38 | func qw_swiglu(inp: *i64, n_tokens: i64, hidden: i64, ffn_dim: i64, Wg: *i64, Wu: *i64, Wd: *i64, out: *i64, hb: *i64) -> i64 |
| 66 | func qw_head_norm(buf: *i64, n_tokens: i64, n_heads: i64, head_dim: i64, gain: *i64, eps: i64) -> i64 |
| 94 | func nx_f32_qwen_layer_qk(x: *i64, n_tokens: i64, hidden: i64, n_q_heads: i64, n_kv_heads: i64, head_dim: i64, ffn_dim: i64, called by 2: nx_f32_qwen_layermain calls 8: sys_mmapnx_i32_to_f32nx_f32_rmsnormqw_linearqw_head_normnx_f32_gqa_attention+2 |
| 145 | func nx_f32_qwen_layer(x: *i64, n_tokens: i64, hidden: i64, n_q_heads: i64, n_kv_heads: i64, head_dim: i64, ffn_dim: i64, |
| 155 | func main() -> i64 |