nx_layernorm.nx

nx_layernorm.nx -- Layer Normalisation (Ba/Kiros/Hinton 2016). Sibling of nx_rmsnorm.nx (shipped dfce1e32). RMSNorm drops the mean-subtract centering for speed; LayerNorm keeps it. LayerNorm is still the choice for many architectures: GPT-2 / GPT-3 / GPT-Neo / GPT-J BERT / RoBERTa / DistilBERT T5 / mT5 / ByT5 ViT / DeiT / Swin / CLIP Whisper, Wav2Vec Modern decoder-only LLMs (Llama, Mistral, Qwen) moved to RMSNorm for the ~50% speed win at equivalent quality; encoder-decoder and older architectures use LayerNorm. The substrate ships both so the model loader picks per-layer. ===== Math ======================================================= For each token i, hidden dim D: mean = sum_d x[i, d] / D var = sum_d (x[i, d] - mean)^2 / D norm = (x[i, d] - mean) / sqrt(var + eps) y[i, d] = norm * gamma[d] + beta[d] gamma is the learned per-channel scale (init Q10=1024). beta is the learned per-channel bias (init 0). ===== Q-format =================================================== All tensors in Q10 (substrate convention). mean accumulates in raw integer (sum of Q10 values). variance accumulates as Q20 (sum of Q10*Q10 differences), divided to Q10 before sqrt. eps_q10 = 1 (tightest Q10 stabiliser, same as RMSNorm). Per the bits-up cardinal: composes NxTensor (L1 canonical container) nx_isqrt_q10 (L2 canonical sqrt)