nx_groupnorm.nx

nx_groupnorm.nx -- Group Normalisation (Wu & He 2018). Closes the missing-dep gap for the UNet block. Modern image-gen architectures use GroupNorm, not RMSNorm or LayerNorm: Stable Diffusion 1.x / 2.x / 3 UNet: GroupNorm(num_groups=32) Flux UNet: GroupNorm Z-Image: GroupNorm StyleGAN family: GroupNorm + AdaIN (queued) Substrate had RMSNorm (decoder-only LLMs) + LayerNorm (encoder- decoders / GPT-2 era). GroupNorm was the third missing norm variant and the load-bearing one for diffusion. ===== Math (Wu & He 2018 _Group Normalization_) ================= Input: x [N, C, H, W] Q10 Groups: G (C must be divisible by G; typically G=32) For each (n, g): mean = mean of x over (C/G channels in group g, H pixels, W pixels) (i.e., (C/G * H * W) elements per group per sample) var = variance over the same elements For each (c in group g, h, w): y[n, c, h, w] = (x[n, c, h, w] - mean) / sqrt(var + eps) * gamma[c] + beta[c] gamma is the per-channel learned scale (length C, Q10). beta is the per-channel learned bias (length C, Q10). Group-count edge cases: G = 1 -> LayerNorm-like (normalize over all C*H*W) G = C -> InstanceNorm (normalize per-channel separately) Standard GroupNorm uses G=32 (Wu & He's recommendation, picked to be roughly invariant across batch size). ===== Q-format =================================================== x, gamma, beta all in Q10 (substrate convention). Accumulators