nx_nofloat_olmoe.nx
buildroot/runtime/nx_nofloat_olmoe.nx
about
nx_nofloat_olmoe.nx -- the faithful OLMoE transformer LAYER in deterministic no-float (MoE rung 4,
2026-07-15). OLMoE-1B-7B blk shape (recon-verified): D=2048, 16 heads x head_dim 128, NO GQA, NO biases,
**QK-norm = RMSNorm over the FULL 2048-dim Q and K projections** (gamma[2048], the OLMoE/Qwen3 variant --
applied AFTER projection, BEFORE RoPE), RoPE base 1e4, then MoE-FFN (64 experts top-8, lazy). Two
sublayers; the caller (gate/serve) does the residual adds + owns all scratch (lean frames, no big-frame
GP-fault). Composes ONLY proven primitives: rmsnorm_gamma_row / mm_out_in / rope_apply / fx_exp / silu /
qmul / dequant_to_q16 / nf_expert_byteoff. Deterministic by construction (exact integer sums, positional
tie-breaks). license_tier: ORIGINAL (lib: no main)
dependencies 2 imports · 2 importers
imports: nx_syscalls.nxnx_nofloat_llm.nx
imported by: nx_nofloat_olmoe_forward_gate.nxnx_nofloat_olmoe_gate.nx
structs
| none |
consts
| none |
functions
| 17 | func nolmoe_attn(ap: *i64) -> i64 |
| 104 | func nolmoe_moe(mp: *i64) -> i64 called by 2: mainmain calls 7: rmsnorm_gamma_rowmm_out_infx_expdequant_to_q16nf_expert_byteoffqmul+1 |