nx_roofline_gate.nx

nx_roofline_gate.nx -- IS DECODE COMPUTE-BOUND OR MEMORY-BANDWIDTH-BOUND? WHY (2026-07-31): an open sev-9 calls the sovereign embedder '~1000x too slow' at 400ms/token and declares the path infeasible. That was read as a CODE defect. But a batch=1 decode step must read EVERY weight once, so the floor is bytes/bandwidth and nothing else: 0.5B params x 4 bytes (f32) = 2.0 GB READ PER TOKEN. At a few GB/s that is hundreds of ms/token BY ARITHMETIC, with a perfect kernel. If measured is near the floor then the code is not slow, it is AT ROOFLINE, and the only levers are FEWER BYTES (4-bit weights) or MORE WORK PER BYTE (batch m>1). Tuning the dot product would buy nothing. THE CODEBASE ALREADY REASONED THIS OUT, INDEPENDENTLY: nx_f32_lazy_weight.nx:706 states 'the 2.72x single-dot gate was L1-resident = misleading. The real decode lever is FEWER BYTES read (keep weights QUANTIZED 4-bit + fast SIMD dequant-dot), or batch tokens (m>1 = compute-bound), NOT f32 packing.' AND IT MATCHES 2026 EXTERNAL SOTA: Neural Speed reports >90pct of MEMORY BANDWIDTH utilisation on INT4 GEMV (the good engines are already AT the bandwidth bound, so bandwidth IS the ceiling), and Graviton3/Llama3-8B-4bit measures 45.5 tok/s at batch=1 vs 184.8 tok/s at batch=8 -- a ~4x win from BATCHING ALONE, which is exactly the signature of a bandwidth-bound decode turning compute-bound. WHY THIS ORGAN EXISTS RATHER THAN A NOTE: a small model will never re-derive this. The insight has to live in an INSTRUMENT that fires for whoever comes next, or it is rediscovered by hand every time -- which is how 400ms/token got recorded as a defect instead of a roofline. nx_roofline_gate -- selftest teeth only nx_roofline_gate est <Mparams> [ms] -- floors for f32/int8/int4; with [ms], the roofline fraction license_tier: ORIGINAL No hw writes (Rule 26). expect_exit: 0