nishi code wiki / SOTA rung plan

The SOTA Rung Plan

How this ecosystem climbs from organically-grown to state-of-the-art, one measured rung at a time. Every number here was measured on 2026-07-31, not asserted.

Coverage is the root class.

Seven distinct failure classes recurred in a single session. Every one has the same shape: a guard existed, and it did not cover the surface where the failure actually happened. Not absent guards — guards whose coverage was never measured.

An unmeasured coverage fraction always drifts toward zero on the surfaces nobody looks at. Gates prove a thing works. Almost nothing here proves what fraction of the live surface has a gate at all — so the uncovered fraction is invisible, and every recurrence lives there.

Deploy plane coverage

10/15

666‰ · was 333‰ · doubled by rows, not code

Independent evidence

0/41

PROVEN · signer registry absent ⇒ every attestation refused

Gates a ruler can read

106/413

256‰ · the rest are correct but invisible

Modules in tree

17870

94589 functions · 46876 import edges

The seven recurrence classes

Each row is a real failure observed on 2026-07-31, traced to the coverage gap that permitted it.

#	class	the guard	what it did NOT cover	evidence
1	Primitive vs callers	cloexec gate proves the shared listen helper	daemons that never call it	3 gateways bind by raw syscall; rebuilding fixes nothing
2	Lib fix vs deployed binary	the fix is in the library	binaries built before it	a lib fix reaches an organ only on its next rebuild
3	Deploy plane vs fleet	validate → health → auto-rollback	10 of 15 running services	coverage 5/15, measured
4	Declared vs real	the inventory route	the allowlist actually read	inventory claims 2 targets the allowlist lacks
5	Liveness vs readiness	port-accept check	whether it serves	:18098 accepted and never sent a ServerHello; 3 instruments said UP/GREEN
6	Remediation vs its own trigger	wedge-kill fires on a detected wedge	whether a wedge was really there	87% of fires kill nothing — a noisy detector, not (as first filed) an impotent killer
7	Instrument vs its own subject	the gate itself	whether it can fail, or even see	a deploy organ failing `127` on both legs; a scanner pid-capped below every daemon it existed to find

Class 7 is the multiplier. An instrument that cannot fail makes every class above it invisible. Two independent lanes hit it on the same day.

Why coverage stalled — the design cause, not the discipline cause

Every deploy allowlist row names a bespoke handler subcommand. Adding a deployable therefore requires writing plumbing, so coverage grows only as fast as someone hand-writes a handler — and stalls at whatever was urgent that month.

When extending coverage costs code, coverage stops at the last urgent thing. Make it cost data.

This is the classic failure mode of organically-grown platforms: they end up held together by point-to-point connections, and more time goes into maintaining integrations than building capability. The industry answer is a golden path — one paved road that turns deployment from an act of heroics into a routine capability.

The ladder

Each rung has an exit criterion measured by an instrument. A rung is not climbed because it feels done; it is climbed when its instrument says so. Rungs are ordered by leverage: each one is the precondition for the next.

GREEN

R0 · CATALOG — one derived inventory

Exit: exactly one answer to “what exists, what kind is it, where is it deployed”. The inventory route is derived from the allowlist, not hand-authored. The organ-kind reader points at a file that exists.

Now: the inventory route derives its list by reading the same allowlist the deploy path reads — 16 targets, declared and real coverage identical by construction. The hand-authored list it replaced had drifted: it advertised two targets the allowlist did not contain, telling callers a fix could be landed that could not. The organ-kind reader is live too, pointing at the surviving declaration file — it had been correct in source for some time but could never ship, because the module did not compile.

The last thing standing was not code but an error message. It still said “where to declare it is currently broken — do not add rows anywhere”, advice that was true when written and now forbids exactly the action that resolves the refusal.

An error message is part of the API, and a stale one misdirects every reader. A message that outlives its defect is worse than no message, because a reader trusts it and stops.

Fixing this exposed why the route had never been corrected: the file did not compile. A botched edit had left two definitions of one function, so the whole module was unbuildable — and “source edited, not built” had been read as caution when it actually meant cannot be built.

A file that does not compile blocks every lane that touches it, silently, until someone builds it.

A derived artifact must not be authorable. The moment a summary can be written by hand, it can disagree with the thing it summarises — and only the smaller number is ever true.

GREEN

R1 · COVERAGE MEASURED — every plane publishes covered/total

Exit: each plane publishes a coverage fraction to a durable record. The headline is the MIN across planes, never the mean — a mean lets a strong plane hide a dead one.

Done. Four planes now write a measured line every time the beat runs, and the beat runs on the clock:

plane	measured
deploy	10 of 15 running services deployable — 5 uncovered, named
gates	107 of 414 ruler-readable — the MIN, so the headline
sockets	130 socket fds, 5 listeners, 3 without CLOEXEC — named
supervisor	1 root, 1 forked keeper

Each is derived at run time and each carries a non-vacuity tooth that goes RED when the scan reaches nothing — because an unread plane and a clean plane must never render the same.

Absence of findings and absence of measurement render identically unless you make one of them RED.

GREEN

R2 · GOLDEN PATH — coverage costs data, not code

Exit: adding a deployable is one row — name, artifact, health port, rollback — with no bespoke handler. Onboarding a legacy service is allowed to be partial: a service may join with a partial scorecard rather than being excluded until perfect.

Now: the root cause is located exactly. cmd_deploy() takes no parameters — it is hardcoded to one set of path constants and to killing one named process. It has a generic name but is the deployer for a single service, which is why every other target needs its own copy. The cost is counted: 15 deploy functions and 12 rollback twins — 27 near-duplicate implementations of one atomic-publish-and-restart pattern.

Done, and proven end-to-end. hc_deploy_one() now carries all four hard-won lessons in one place — refuse-before-destroy, print the real rc, accept either staged name, consume both on success — and gdeploy@<row> reads its live-path and kill-name from the same row the deploy API already reads, so there is no second file to keep in sync.

The proof: two daemons with no deploy path at all were deployed through the sanctioned API — validate → promote → health → auto-rollback → DEPLOYED-GREEN — by adding rows and zero code. Rollbacks banked, both services back up, and the coverage gate measured its own improvement: 333‰ → 666‰, doubled.

Resolve the path a daemon RUNS FROM, never the tree you built in. Three different roots are in play here; a promote to the wrong one succeeds and reaches nothing. Two live services were even found running deleted inodes — executing binaries that no longer exist on disk — which only readlink /proc/PID/exe reveals.

Count the duplicates before arguing about the abstraction. “Each target needs bespoke plumbing” is an opinion; 27 copies of one pattern is a measurement, and it is the argument.

GREEN

R3 · CONTINUOUS — the instruments run unprompted

Exit: coverage and gates run on a beat that nobody has to remember. Drift is treated as a rate over a window, not a series of one-off surprises.

Done, and watched. nx_coverbeat is registered with the consolidated tickless scheduler and fired on its own — no human involved — writing one line that carries its own evidence: attempted=3 reaped=3 unrunnable=0 worst=0 VERDICT=GREEN. An operator can tell ran 3, all green from ran 0, reported green without opening anything else.

It is proven in both directions: with the gates present it exits 0; run where they do not exist it exits 3 and says reaped ZERO instruments. And it uses the scheduler that already existed rather than adding a parallel one — the adoption law applied to scheduling.

fail=0 is not evidence that a beat moved what it names. A scheduled job that reports success without doing work is the quietest failure in the system.

AMBER

R4 · NON-VACUOUS — every gate can fail, and proves it looked

Exit: every gate carries (a) a negative control — a case that must fail, and fails today — and (b) a did-I-measure-anything tooth that goes RED when the scan reached nothing. Verdicts are cross-checked against what was logged, not just what was printed.

Now: the pattern is proven and adopted in this session's gates — all verified AGREE by an independent verifier that runs them and compares printed against logged. The fleet-wide half is now measured rather than guessed: of 413 promoted gates, 127 carry a parseable pass ratio, 181 carry a durable log path, and only 106 carry both — 256‰. Three quarters of the gate fleet is correct but invisible to any rollup.

A gate that cannot be read contributes nothing to any rollup, however correct it is. Four such gates shipped in this very session before the gap was noticed — which is why the number is measured on a beat now, not remembered.

A fix is not proven by the gate that passes; it is proven by the case that now fails and used to succeed.
A scan bound is a claim about the population — take it from the system, never from a round number.

GREEN

R5 · SELF-HEALING — remediation targets reality

Exit: remediation targets the actual holder of a resource (the kernel's FD table), never an expected name. A remediation that changes nothing escalates. Recovery tooling is reachable when the control plane is down.

Proven end to end, on a real fault, with no human in the path. The beat fired on the clock and went RED. The singleness gate named three supervisor roots — all three ppid=1, so three independent trees, not a parent and its children. The sentinel had already detected it and was correctly suppressing under its cooldown, because a remedy that runs every minute becomes the churn. At t+240s the cooldown expired, it reconciled, and roots went 3 → 1.

The circularity is broken too: four recovery caps are pre-minted and exercised, so diagnosis no longer depends on the control plane being up.

Detection was the gap, not remediation. The remedy was already correct and already scheduled; what was missing was an instrument that could see the condition and a beat that ran it.

And the instrument earned its keep by staying quiet: an earlier version falsely called a healthy supervisor-plus-keeper a duel, and two debts were filed on that misreading before anyone checked parentage. A detector that cries wolf on the healthy shape is worse than none.

Three observations of the common case are not proof of impossibility. To claim a mechanism cannot work, read its whole distribution, not a burst — this correction retracted a sev-9 root cause.

killed=0 is a failed remediation — escalate, never print green.
A control plane's recovery tooling must not be gated by the control plane. If the only way to get authority to fix X is through X, then X has no remedy.

RED

R6 · INDEPENDENT EVIDENCE — someone other than us says it works

Exit: the proven count moves off zero against oracles we did not author — published test vectors, external reference implementations, signed third-party attestation.

Now: 0 of 41 — and it is structurally guaranteed, not a shortage of evidence. The signer registry attest_keys.conf does not exist, and the attestation loader states the rule in its own source: an absent registry means every key is unregistered, therefore every attestation is refused. That is the correct fail-closed direction, but it means no amount of new testing moves the number.

The real shape is better than the headline: 7 domains are already MECH-OK — quorum, provenance and freshness all hold, awaiting only a second independent method class. And genuinely independent mechanical evidence already exists: byte-exact agreement with a published NIST FIPS-203 vector, RFC5903, and a TLS-WG vector — oracles authored by people who never saw this implementation.

An oracle row must be derived, never declared. “KAT” names the form, not the provenance — a test vector is only independent if its author never saw your implementation.

This rung cannot be climbed by the agent doing the work. Creating the signer registry and signing its own rows would be a system granting itself attestation authority, which makes the number meaningless by construction. Deciding who is authorised to attest is an operator act — and the fact that it is blocked here is the control working, not failing.

What good looks like, per dimension

Mapping the ladder onto the five dimensions the platform-engineering field uses to score maturity.

dimension	ad-hoc (where organic growth lands)	target	our rung
Investment	whoever is on fire this week	a standing owner per plane	R3
Adoption	capabilities exist; nothing points at them	the paved road is the easy road	R2
Interfaces	bespoke handler per target	one declarative contract	R2
Operations	restart and hope	remediation targets reality; no-ops escalate	R5
Measurement	green dashboards over a dead service	coverage per plane, headline = MIN	R1, R4

The adoption trap

The bottleneck is adoption, not primitives. This ecosystem repeatedly builds a capability that already exists, because the existing one was never discoverable at the point of use. Discoverability is the capability — a tool nobody can find has the same value as a tool nobody wrote, and costs more.

Concretely: a gate proven correct but not wired at the live chokepoint is the baseline, not the improvement.

How to read this page

Rung status is deliberately harsh: five GREEN, one AMBER, one RED — on a system that demonstrably works and ships. That is not a contradiction — it is the point. The ecosystem is capable and under-measured, and the second fact is what makes the first fragile. Every RED here is a known, named, filed gap with an instrument that can confirm when it closes.