nx_nofloat_scale_gate.nx

STATUS 2026-06-23: PARTIAL / NOT REGISTERED -- HONEST documented attempt mapping the frontier. On the richer 4-category grammar (vocab 16) at dm=32, a SINGLE-BLOCK no-float LM SCALES and LEARNS STRUCTURE but does NOT reach optimality at tractable compute: - low-lr clipped SGD (0.1): plateaus at uniform (CE ~2738 ~= ln16 2773) -- too-small steps. - higher-lr clipped SGD (0.2): ESCAPES the plateau, CE ~2135 (perplexity ~8.5) = clearly BELOW uniform (2773 / ppl 16) so it learned the category structure, but still well ABOVE the floor (1324 / ppl 3.76) = NOT near-optimal. lr 0.3 / more steps did not improve (plateaus ~2135-2295). THIS IS THE BEST RESULT. - lr DECAY (0.2->0.02): WORSE (CE 2510) -- the low late-lr can't progress; not the fix. - AdamW: diverges at lr>=0.01 / stalls at uniform at lr<=0.003 on this model (a narrow unstable band). NOTE: I hypothesized this was a Q16 g^2-UNDERFLOW and built a Q32-moment Adam to fix it -- but a CONTROLLED small-gradient fit CONVERGES under BOTH Q16 and Q32 moments (loss 112 vs 85), so that hypothesis is NOT confirmed; the transformer-specific divergence cause is NOT isolated. (over-claim retracted; gate deleted.) - DEPTH-2 (two stacked blocks, SGD 0.2): CE 2236 ~= single-block 2135 -- DEPTH DOES NOT HELP (tested, then deleted). So capacity/depth is NOT the bottleneck. So R4-FULL = PARTIAL, and the boundary is now precisely located: NOT a missing mechanism, NOT capacity/depth (depth tested), but fixed-point OPTIMIZATION reaching sharp output distributions + compute -- a novel-optimizer / much-more-compute investment (operator-gated). The 3-category LANDING (nx_nofloat_landing_gate) IS near-optimal and stands. (don't-lie: not registered, NOT claimed GREEN -- near-optimal genuinely fails here.) nx_nofloat_scale_gate.nx -- R4 SCALE-UP (the frontier step): grow BOTH the language and the model, and show the no-float LM stays NEAR-OPTIMAL. Richer grammar (4 categories, vocab 16): DET(2) -> ADJ(4) -> NOUN(5) -> VERB(5) -> DET ... (e.g. "the big cat ran a red dog sat"). Bigger model dm=32. Fresh-random members each stream -> the model learns the GRAMMAR, measured by HELD-OUT perplexity vs the richer information floor. Floor = avg(ln2,ln4,ln5,ln5) = 1.324 nats (perplexity 3.76; members are random within category so unbeatable); uniform baseline = ln(16) = 2.773 nats (perplexity 16). T1 held-out CE << uniform (learned the richer language). T2 held-out CE ~= the richer floor (near-OPTIMAL at scale = the model scaled up cleanly). Pure integer Q16. HONEST: richer but still a CONSTRAINED domain (open multi-topic prose is the remaining compute frontier). Sovereign: nx_nofloat_autograd + nx_syscalls. expect_exit: 0