Model-tier routing: what 239 real sessions actually measured
Mined 2026-07-29 (all 239 main-session transcripts 2026-06-29→07-29, per-model attribution, usage deduped by message id); corrected same day by the operator; transcribed 2026-08-01. Corpus slug: project-nishi-model-tier-transcript-census-2026-07-29. All numbers are OURS — measured on our own transcripts, not vendor benchmarks.
The finding that matters most is a CORRECTION. The first-pass headline (“the newer premium tier had the lowest tool-error rate: 3.6%”) was WRONG as a quality claim, and the operator caught it: tool-error rate measures shell fumbling, not work quality — bad code that compiles fires no error; mixed-model sessions attribute a model's damage to whoever repairs it later; and the “clean” segments ran inside context scaffolded by the stronger tier. On real quality proxies the ranking INVERTED: revert/redo language per 1000 blocks = 19.7 (the “clean” tier) vs 8.1 (the workhorse tier). Law: the author of bad work is not the model in whose segment the failure surfaces.
This census is the incident that commissioned that sweep; its designs (LOOK gate, trajectory detectors, judge hygiene) are the remedies for the failure signature below.
The failure signature of an unsupervised down-tier seat (verbatim case study, 07-27)
1
Gate-satisfaction without LOOKING: “nine gates green” on renders that were visibly broken — optimized the proxy, never opened a pixel until called out.
2
Re-hit DOCUMENTED gotchas at operator expense (/tmp wiped, WSL var-eating, CRLF, sed regex) — lower tiers don't PULL the gotcha corpus; it must be PUSHED into their prompt.
3
Destroyed data via an argument-order guess — didn't read usage before a destructive CLI.
4
Contrition spirals and hedge-framing burning turns on framing instead of fixing.
5
Mid-session tier switching = a cheap model writing bad output INTO expensive context, which the expensive model then re-reads and repairs — paying twice.
The routing laws (standing, enforced by data not memory)
1
ENUMERABLE-GATE ONLY: down-tier iff the gate refuses wrong results AND the failure mode is enumerable (bit-identity, byte counts, exit codes). Perceptual/quality-shaped outputs (renders, meshes, UI, prose) are NEVER flash-class — a numeric gate cannot see shape.
2
NEVER switch tiers mid-session. Down-tier work goes to fresh COLD seats with a recipe; premium sessions stay premium.
3
PUSH gotchas into executor prompts — the scaffold is sovereign data (capuplift- plane; a new law = one row, no rebuild).
4
A LOOK step by construction for any visual output: render → read the image → describe → only then claim. Read usage/help before any destructive CLI.
5
Executor discipline: no narration, no apologies — state defect + fix in ≤2 sentences, then act.
6
Lagging truth, not vibes:nx_trajscan beats hourly (verification-skip · search-loops · oracle-edits · retry-echoes) — the number the scaffold discipline should bend down.
Standing numbers (three tiers, our transcripts)
metric
workhorse premium
frontier
newer premium
sessions
176
170
36
tool calls
75.6k
36.8k
10.4k
revert/redo language /1000 blocks
16.6
8.1
19.7
prose chars per tool call (narration tax)
292
94
182
PowerShell per-call error
14.7%
12.1%
7.4%
Declared UNVERIFIED / caveats
1
Reverty-language partly counts cleaning up OTHERS' messes; the smallest tier sample is 36 sessions over 5 days. Direction confirmed by operator verdict + the case study, not by the proxies alone.
2
Census scripts are scaffolding (_ops/claude_census/), re-run monthly; numbers here are the 07-29 snapshot and drift with model releases.
3
Model names are deliberately tier-generic on this page; per-model identities and the full incident trail live in the corpus slug.