code wiki / (root) / nx_nofloat_olmoe.nx

nx_nofloat_olmoe.nx

buildroot/runtime/nx_nofloat_olmoe.nx

7629 B185 linesdepth 8pulls 16 transitivereach 2 importersview sourcekind librarytopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_olmoe.nx -- the faithful OLMoE transformer LAYER in deterministic no-float (MoE rung 4, 2026-07-15). OLMoE-1B-7B blk shape (recon-verified): D=2048, 16 heads x head_dim 128, NO GQA, NO biases, **QK-norm = RMSNorm over the FULL 2048-dim Q and K projections** (gamma[2048], the OLMoE/Qwen3 variant -- applied AFTER projection, BEFORE RoPE), RoPE base 1e4, then MoE-FFN (64 experts top-8, lazy). Two sublayers; the caller (gate/serve) does the residual adds + owns all scratch (lean frames, no big-frame GP-fault). Composes ONLY proven primitives: rmsnorm_gamma_row / mm_out_in / rope_apply / fx_exp / silu / qmul / dequant_to_q16 / nf_expert_byteoff. Deterministic by construction (exact integer sums, positional tie-breaks). license_tier: ORIGINAL (lib: no main)

dependencies 2 imports · 2 importers

nx_syscalls.nx nx_nofloat_llm.nx nx_nofloat_olmoe.nx nx_nofloat_olmoe_forward_gate.nx nx_nofloat_olmoe_gate.nx

imports: nx_syscalls.nxnx_nofloat_llm.nx

imported by: nx_nofloat_olmoe_forward_gate.nxnx_nofloat_olmoe_gate.nx

structs

none

consts

none

functions

17func nolmoe_attn(ap: *i64) -> i64
104func nolmoe_moe(mp: *i64) -> i64