Nishi Family › Compare › Traffic Safety
Nishi Compare · measured, not asserted
Traffic Safety
Nishi vs the field — every Nishi cell is measured against real organ source at emit time; each gap names the watch contract that will close it.
How traffic survives a change and how a bad change is undone -- the sovereign edge, routing and process replacement vs nginx and HAProxy and Envoy and Argo Rollouts. Rival codes are documented-capability reads of 2026-08-21, each grounded by a mirrored and content-pinned reference. Nishi cells are MEASURED -- the generator requires the implementing symbol in the real organ source on disk.
Where we are. Measured 2026-08-21, full population with declared coverage. We hold the half of this domain nobody else has -- two capability-loss oracles that name every route a candidate binary would stop serving, built after that regression hit us five times -- and we are behind on every mechanism that keeps traffic alive while a change lands. Three numbers set the position. nx_http_server_listen_hot resolves 9 times estate-wide and exactly 2 are production call sites, so the sites.elf front door that all 69 live route rows cross cannot hand off at all. rt_sigaction resolved 8 times and every one was SIGPIPE handling or a crash handler or the signal library itself, so ZERO daemons drained on TERM -- and that is now ONE, nx_survey_daemon, shipped 2026-08-21. The census also pointed at a file that could never have carried the fix: nx_signal spells rt_sigaction as RV64 134, which is not a key in the x86 backend's translation table whose default branch is a pass-through, so on the live target it lands on uselib and installs nothing. The shipped drain installs no handler at all -- it BLOCKS TERM and consumes it as a pollable descriptor, which removes the EINTR mid-response truncation a handler would itself have introduced, and its deadline is DERIVED from ACCEPT_TMO_S rather than typed. sys_sendmsg WAS ABSENT-PROVEN at corpus_complete=1; that gap was closed on 2026-08-21 and TS1 has since MET its accept rule (ARMED zero drops over 17 connections with the in-gap arrival served, NEG-CONTROL 1 drop with the in-gap arrival refused ECONNREFUSED), so the remaining deficit on the continuity half was the ADOPTION of that mechanism by the serving daemons rather than its absence from the shim -- and that half opened and closed on the same day. nx_survey_daemon is the FIRST ADOPTER: chosen from the route table rather than from taste (one of 69 live rows, its own binary, no control-plane path, deliberately NOT sites.elf which is TS3), it calls ts_handoff_nodrop instead of binding and arms the TS2 drain before its listener exists. Proven in DROPPED CONNECTIONS across a real fork+exec restart of the real binary rather than by a source reference: nx_survey_handoff_gate 12/12, ADOPTED attempted=17 served=17 drops=0 with the in-gap arrival served, NEG-CONTROL the same binary with no owner attempted=17 served=16 drops=1 refused ECONNREFUSED. Envoy's ordering rule went in with it -- the arenas and the seg-store warm now run BEFORE the socket is acquired. What is left is a deploy decision and an owner process, and because the call falls back to binding when no owner publishes, the adopted daemon is safe to deploy before any owner exists. The undo path for a bad deploy runs through the front door whose restart is the incident, and an 8-minute estate-wide blackout is on record.
Where we need to go. A deploy that no client can observe, and an undo that is a process we did not kill rather than a build we have to run. Concretely: the listening socket is never released, the outgoing process keeps serving after TERM until a named deadline, a candidate is exercised on mirrored traffic before any user depends on it, promotion waits for a minimum number of real requests, and the rollback action is mechanically PROVEN not to traverse the thing it rolls back.
Research bar. nginx is measured on both generations accept at once and rollback is a signal to a master that never stopped. Theirs: the continuity bar and the rollback bar. Ours: TS1 and TS7 are measured against this page.
Research bar. HAProxy is measured on the listening descriptor is transferred over SCM_RIGHTS rather than re-bound. Theirs: the only implementation that removed the race instead of shrinking it. Ours: TS1.
Research bar. Envoy is measured on all expensive initialization happens BEFORE the handoff while the old process still serves. Theirs: the ordering rule that is the direct fix for our blackout. Ours: TS1 and TS2.
Research bar. Argo Rollouts is measured on metric-gated promotion with a genuine third inconclusive state that pauses. Theirs: the progressive-delivery reference. Ours: TS5 and TS6.
16 of 23 capabilities measured|1 of them measured exceeds|7 open|coverage 695/1000|adoption 13 full / 3 partial
Do this next — computed by the ranker, never chosen by a seat
Order from nx_compare_rank (nx_dr_ocm: (deficit + cost-of-delay + option + enables) x sponsor x self-sufficiency x momentum / cost). FINISH rows are rungs whose symbol is present but whose organ is short of full adoption: the cheapest closures on this board, listed before any new work. Stamp: # asof=1787905107 domain=trafficsafety target_version=1.0 rungs=11 done=4 open=7 finish=0 ranker=nx_dr_ocm
| # | Stage | Rung | Priority | Derivation |
|---|---|---|---|---|
| #1 | 1.0 | Bake with a minimum request denominator (TS5) ts_bake_window | 3000 | v=3 m=2 c=2 |
| #2 | 1.0 | The undo path does not traverse the subject (TS7) ts_undo_independence | 1000 | v=1 m=2 c=2 |
| #3 | 1.0 | Mirror to a shadow port and discard the response (TS4) ts_mirror_shadow | 600 | v=6 m=2 c=20 |
| #4 | 1.0 | Rollback fired by the capability-loss oracle (TS6) ts_rollback_on_caploss | 400 | v=2 m=1 c=5 |
| #5 | later | Wait hints derived from the real bound (TS9) ts_retry_after_derived | 3000 | v=3 m=1 c=1 |
| #6 | later | Request-path admission and shedding (TS8) ioa_request_admit | 333 | v=5 m=1 c=15 |
| #7 | later | Tell the peer what was processed (TS10) ts_goaway_lastid | 333 | v=5 m=2 c=30 |
Critical path — contract, done-rule, executor, cost
| Rung | Closes with | Definition of done (pre-declared) | Executor | Est. |
|---|---|---|---|---|
| Capability-loss oracle on promote (TS0) | rg_missing | Every route the live binary serves is extracted and each one the candidate does not carry is NAMED before promote -- LANDED, and the exceed this domain holds | Organ | 0 u |
| Keep the socket replace the process (TS1) | ts_handoff_nodrop | An owner outside the serving process holds the listener and hands it to each generation, so the port never disappears and arrivals queue rather than being refused. ACCEPT RULE PRE-DECLARED, and it is the estate's own already-written bar: a swap under a synthetic request stream drops ZERO connections, measured by a client that counts refusals and resets, not by the daemon's own log. The neg-control is mandatory -- the same harness against the CURRENT ordinary-listener path must show a non-zero drop, or the test proves nothing. Blocked on sys_sendmsg for the SCM_RIGHTS form, so the socket-activation form ships first and the blocker is named in the row | Organ | 2 u |
| Cooperative drain on TERM with a named deadline (TS2) after TS1 | ts_drain_on_term | On TERM the daemon STOPS ACCEPTING and KEEPS SERVING in-flight requests until a declared deadline, then exits; the deadline is a named constant, never a literal. ACCEPT RULE: with a request in flight at the moment TERM lands, the client receives a complete response and not a 5xx or a reset, and a daemon that exits early FAILS the gate. Anti-vacuity tooth required -- a run in which no request was in flight when TERM landed must be reported as UNOBSERVED and must not score as a pass | Organ | 1 u |
| Front door that never restarts during a deploy (TS3) after TS1,TS2 | ts_slot_flip | Two backend slots on two ports behind one small front door that owns the upstream choice and is independent of both. ACCEPT RULE: a flip changes which slot answers with zero failed requests across the flip, a slot that fails its warm-up probe is NEVER flipped to, and the front door's own uptime is unbroken across the whole exercise. The front door is the only component in this domain that must never restart, so its own replacement is TS1's problem and is a stated dependency, not an afterthought | Organ | 1.5 u |
| Mirror to a shadow port and discard the response (TS4) after TS3 | ts_mirror_shadow | Each incoming request is copied to a shadow instance whose response is discarded and never reaches the client. ACCEPT RULE: over a declared number of mirrored requests the client-visible responses are byte-identical to the un-mirrored control, the shadow's responses are proven discarded, and a shadow that returns errors or hangs changes NOTHING the client sees -- proven by killing the shadow mid-run and observing no client-visible effect | Organ | 2 u |
| Bake with a minimum request denominator (TS5) after TS3 | ts_bake_window | A promoted candidate is held under observation for a declared window that advances only after a declared minimum number of real requests. ACCEPT RULE: a window that saw fewer than the minimum reports UNOBSERVED and REFUSES to promote rather than passing -- the gate-passes-on-the-empty-set defect is the specific thing this rung exists to make impossible -- and the third state is distinct from both pass and fail in the exit code, not only in the text | Organ | 1 u |
| Rollback fired by the capability-loss oracle (TS6) after TS5 | ts_rollback_on_caploss | The oracles that already exist drive an automatic revert to the previous process. ACCEPT RULE, all four required and each with its own tooth: a two-window burn-rate signal whose short window demonstrably stops the action latching after recovery, a bounded number of automatic actions per unit time proven by a flapping-input fixture that must NOT produce unbounded reverts, a minimum-request denominator, and a time-boxed off switch that AUTO-RE-ARMS -- proven by disabling it and observing it re-arm without a human. A flapping input that drives repeated reverts is a RED, because that failure mode is the one that caused the outage this rung cites | Organ | 2 u |
| The undo path does not traverse the subject (TS7) after TS3 | ts_undo_independence | A gate resolves the dependency chain of the rollback ACTION -- which process serves it, which route reaches it, which daemon authenticates it -- and REFUSES when any link passes through the component being rolled back. ACCEPT RULE: run against TODAY's topology it must go RED and NAME sites.elf, because the undo route demonstrably crosses the front door; run against a topology where the undo path is independent it must go GREEN. A gate that cannot go RED on the current estate is vacuous by construction and must not be shipped | Organ | 1 u |
| Request-path admission and shedding (TS8) | ioa_request_admit | The existing shared admission ruler gains a request-path verdict so a saturated box sheds EARLY rather than accepting work it cannot finish. ACCEPT RULE: shedding is proven to preserve goodput against an un-shed control at the same offered load, and the shed response is a non-retryable error class rather than one that invites a retry. Extends the incumbent ruler -- a second admission ruler is refused by construction | Organ | 1.5 u |
| Wait hints derived from the real bound (TS9) | ts_retry_after_derived | Any advertised wait is derived from the SAME named constant as the resource it waits on, emitted as a machine-readable header, and jittered on emission. ACCEPT RULE: changing the lock TTL constant changes the advertised value with no second edit -- proven by changing it and re-reading the emission -- and a resource with no bounded recovery advertises a non-retryable class and NO wait hint at all | Organ | 0.5 u |
| Tell the peer what was processed (TS10) after TS2 | ts_goaway_lastid | Graceful shutdown announces the last request the server may have acted on so the client can decide retry-safety instead of guessing. ACCEPT RULE: a client that receives the announcement retries exactly the requests after that marker and no others. SCOPE DECLARED UP FRONT: the mechanism is HTTP/2 GOAWAY and our edge serves HTTP/1.1, so this rung carries a protocol prerequisite and is priced with it rather than promised without it | Organ | 3 u |
Milestones
| Milestone | Rungs | Cumulative |
|---|---|---|
| M1 · A deploy nobody sees | TS1,TS2,TS3 | 4.5 u |
| M2 · A canary that serves nobody and an undo we can trust | TS4,TS5,TS6,TS7 | 9.5 u |
| M3 · The request path and the protocol | TS8,TS9,TS10 | 14.5 u |
comparewatch- plane row flips with it. The flip is necessary, not sufficient: it proves the symbol exists, never that the capability is good. The bar is the rung's pre-declared done-rule, proven by its gate — a symbol shipped without the behaviour behind it is a defect, and the flip is exactly what makes that defect visible instead of quiet. Competitor marks record documented capability presence — presence, not depth or scale. Adoption is measured too: every measured row carries where its organ stands on the estate's ladder (source → built → promoted → registered → invoked; libraries by importer reach minus validation importers; gates by the execution surfaces that run them). A row is fully adopted only at the top of its ladder; anything short is tagged partial with the exact remedy, so a build nobody promoted can no longer read as shipped. Census stamps: importers asof 1787849099, gate census asof 1787855507 (unix seconds; -1 = census absent).Capability matrix — measured against source
◉ leads / measured exceed● present◐ partial○ absent · click any capability for its evidence
| Capability | Nishi | nginx | HAProxy | Envoy | ArgoRollouts |
|---|---|---|---|---|---|
Hot listener a second process can hold the port while the old still servesMeasured:nx_http_server_listen_hot exists in runtime/nx_http_server.nx, verified at emit. nginx inherits the descriptor across fork and both generations accept at once [nginx-binary-upgrade]. HAProxy transfers the actual listening file descriptors over SCM_RIGHTS so the socket is never closed and never re-bound [haproxy-seamless-reload]. Envoy asks for copies of the listen sockets only after the new process has fully initialized [envoy-hot-restart]. Ours is SO_REUSEPORT co-binding, and the mgmt API's own adoption comment records nx_hotlisten_gate at 3 of 3 with T1 as the non-vacuity control -- a standard listener REFUSES the second bind, so the disease state is proven reachable before the cure is credited. HONEST SCOPE AND IT IS THE POINT OF THIS PAGE: LWN documents SO_REUSEPORT as carrying a DEFECT that drops connections during the three-way handshake when the listening set changes, so this proves the NECESSARY condition of two binders and never the sufficient one of zero drops [so-reuseport-lwn]. ADOPTION MEASURED FULL POPULATION at corpus_complete=1: 9 references estate-wide of which exactly 2 were production call sites when this page was admitted, nx_mgmt_api and nx_tools_api_serve, and BOTH of those hold a SO_REUSEPORT co-bind rather than a handoff. As of 2026-08-21 there are THREE, and the third is the only one that is a true HANDOFF: nx_survey_daemon calls ts_handoff_nodrop, proven in dropped connections across a real restart by nx_survey_handoff_gate 12/12. So the count understates the change and the KIND is the part that matters. The sites.elf front door every request crosses is still not one of them, and that is rung TS3 rather than an oversight. Adoption: LIB-WIRED importers=64 nonval=52 — fully adopted (top of its ladder). | ● | ◉ | ◉ | ◉ | ○ |
Listener cannot be captured by a forked childMeasured:cx_is_cloexec exists in runtime/nx_cloexec_gate.nx, verified at emit. Five teeth with T1 as the CONTROL -- a raw socket must read the FD_CLOEXEC bit CLEAR, so a kernel that set it by default could not fake a pass -- and a neg-control on the reader itself. THE OUTAGE IT EXISTS FOR, measured live 2026-07-30: /api/deploy forked nx_hostctl from inside nx_mgmt_api while mgmt held its listening socket with no FD_CLOEXEC, hostctl inherited the descriptor, and being the long-lived supervisor it held mgmt's port FOREVER and re-leaked it into every child it spawned -- an outage that SURVIVES EVERY RESTART and reads as a causeless crash-loop. DELIBERATELY NOT CLAIMED AS AN EXCEED: the three proxies never re-bind, so a child capturing the listener is not a hazard their design can produce, and absence of the test is not absence of the property. AND IT BOUNDS THE ROW ABOVE -- the kernel permits co-binding only when EVERY socket on the port set SO_REUSEPORT, so one inherited legacy socket locks out even a REUSEPORT binder. Adoption: GATE:LIVE trial=GREEN — fully adopted (top of its ladder). | ● | ○ | ○ | ○ | ○ |
Promote refused when the candidate serves fewer routes than liveMeasured exceed:rg_missing in runtime/_hdl_build/nx_routeguard_lib.nx, verified at emit. EXCEED. THE FIELD ROLLS BACK ON AVAILABILITY PROXIES AND NOTHING ELSE -- Argo aborts on metric success conditions [argo-rollouts-analysis] and Google's canary service states the limit outright, asking whether the canary is meaningfully worse and NOT trying to establish absolute health, with good health metrics surprisingly rare [cas-canary-analysis]. A build that answers 200 to everything while having silently stopped serving /api/promote is invisible to every one of the four columns. rg_missing extracts every route literal from the LIVE binary and NAMES each one the candidate does not carry, before the promote. Built after the regression happened five measured times, one of them the author's own. Adoption: LIB-WIRED importers=3 nonval=2 — fully adopted (top of its ladder). | ◉ | ○ | ○ | ○ | ○ |
Route-set shrink detected across a deploy transactionallyMeasured:rd_in_baseline exists in runtime/_hdl_build/nx_route_diff.nx, verified at emit. The DETECTION half of the same capability and deliberately NOT a second exceed claim -- snapshot the served route set before a deploy, check after, and ANY route absent versus the captured baseline is named. It complements rg_missing which compares BINARIES before promote where this compares the SERVED SET after, so the two answer different questions about the same defect. REPOINT NOT DUPLICATE: /compare/failmodes owns the general capability-loss oracle as cd_contains and this row is the routing instance of it. The class it was built for is concrete -- routes such as /api/promote and /tools/register and /cap/mint silently vanishing on a deploy, nine debt rows in one week. Adoption: RUN-BY:cron — fully adopted (top of its ladder). | ● | ○ | ○ | ○ | ○ |
Two daemons listening on one port detectedMeasured:pd_scan exists in runtime/_hdl_build/nx_portdup.nx, verified at emit. The wrong-answer machine: on 2026-07-30 nx_mgmt_api AND nx_opaque_login were both LISTENING on 127.0.0.1:18098 and the kernel handed each connection to whichever accepted first, so the service answered correctly some of the time. AND THIS DOMAIN MUST NAME THE TENSION RATHER THAN INHERIT IT: a CORRECT hot handoff is two processes listening on one port BY DESIGN, so this detector and the mechanism in the first row disagree by construction. The organ already knows half of it -- its own header records that the neighbouring Synology nginx runs SO_REUSEPORT worker pools showing eight legitimate holders per port -- but nothing yet distinguishes a handoff overlap from a squat, and that discrimination is owed by rung TS1. Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, no MCP invocation on record (a direct fork logs the runner, so this is not proof it never ran); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked). | ● | ● | ● | ● | ○ |
Listening-versus-claimed audit for every declared daemon portMeasured:pa_audit_conf exists in runtime/nx_port_audit.nx, verified at emit. Parses /proc/net/tcp for sockets in LISTEN state and checks each port declared in knowledge/status/portmap.conf: is something ACTUALLY listening, or does the pidfile merely CLAIM up. This is the detector for the open deploy bind race that has twice left a service down with a HEALTHY binary. Coded zero across the field for a stated reason and not by oversight -- none of the three proxies can have this defect because none of them closes the listening socket, so the honest reading is that we built a detector for a hazard our design creates and theirs does not. The remedy is the row above this one, not a better detector. Adoption: LIVE — fully adopted (top of its ladder). | ● | ○ | ○ | ○ | ○ |
Liveness probe whose every failure names its own failing conjunctMeasured:hp_run exists in runtime/nx_http_probe_lib.nx, verified at emit. Thirteen distinct exit codes -- connect-failed, pattern-absent, status-not-2xx, recv-timeout, send-failed, body-truncated, empty-response, not-http and more -- so a compound health assertion cannot fail without saying WHICH half failed, which is the difference between a verdict and a diagnosis. Envoy is Best on health-checking depth. It also stamps a dead-man heartbeat so the PROBE's own failure is observable, and it retired a curl-and-grep cron one-liner that was the estate's only compensating control over a fail-open authentication path and whose own failure was structurally unobservable. Adoption: LIB-GATE-ONLY importers=2 — PARTIAL: imported only by validation organs (gates, tests, benches): wire it into a shipping program. | ● | ● | ● | ◉ | ● |
Public-site liveness verdict written where a rollup reads itMeasured:sl_check exists in runtime/_hdl_build/nx_site_liveness.nx, verified at emit. Probes each public site for HTTP 200 plus a body plus the X-Served-By edge marker over the sovereign fetcher, and appends verdict GREEN or RED to a log the maturity rollup reads. Presence, not depth, and the wired-state is stated rather than implied: the organ is REGISTERED-DARK -- promoted, registered, authorised, with no invocation on record. Adoption: LIVE — fully adopted (top of its ladder). | ● | ● | ● | ● | ● |
Reachability measured from a second vantage that does not share fateMeasured:ev_has exists in runtime/_hdl_build/nx_edge_vantage.nx, verified at emit. AWS states the requirement plainly -- monitor from a second adjacent Region, which is more representative of the end user's experience and reduces the risk of the application and its monitoring being impaired by the same event [arc-dataplane-recovery]. Ours probes the public edge from off-LAN over our own TLS 1.3 stack, and that matters here because PORT REACHABILITY IS A PROPERTY OF THE VANTAGE and not of the host: on-LAN 443 is closed and 8443 open, off-LAN the reverse, so a single-vantage check calls a healthy host down. NOT AN EXCEED AND THE ZEROES SAY WHY -- the four columns are three proxies and a rollout controller, none of which is a monitoring product, so a zero here is a scope statement and not a lead. Cloudflare and AWS do this and are deliberately outside this column set. Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, no MCP invocation on record (a direct fork logs the runner, so this is not proof it never ran); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked). | ● | ○ | ○ | ○ | ○ |
Route table is data rather than a recompiled branchMeasured:pr_lookup exists in runtime/_hdl_build/nx_proxy_route.nx, verified at emit. 69 live rows over 4 hostnames and roughly 35 backend ports map host plus path-prefix to a port and a mode, longest-prefix wins, exact host beats the wildcard, and the prefix test is segment-boundary-safe so /gen never matches /generated. All three proxies are Best -- this is their core competence. TWO SAFETY FACTS READ OUT OF THE LIVE FILE. First, mode=gated means fail-closed 302 to /login when the BACKEND IS DOWN and is NOT authentication, so routing an unauthenticated daemon as gated publishes it. Second and sharper: pr_mode recognises only stream and gated and returns BUFFERED for every other token, so a mistyped fail-closed keyword silently downgrades a gated route to buffered with no diagnostic -- and the live table already contains one row whose mode field is a bare 0 rather than any of the three documented names, which is the witness that non-vocabulary tokens do reach this file. Adoption: LIB-WIRED importers=2 nonval=1 — fully adopted (top of its ladder). | ● | ◉ | ◉ | ◉ | ○ |
SNI routing without terminating TLS and malformation falls to a defaultMeasured:sni_backend exists in runtime/nx_sni_route.nx, verified at emit. Pure -- no syscalls and no imports -- so it is unit-gateable offline against hand-crafted ClientHellos, and every offset into untrusted network bytes is bounds-checked with any malformation routing to the DEFAULT backend rather than refusing, which is fail-SAFE rather than fail-closed and is the right choice for a front door. All three proxies do SNI routing and are Best. REPOINT: /compare/deploy and /compare/authz already carry this organ as the sovereign-TLS exceed row, and here it is scored only for its ROUTING safety property, which neither of those pages measures. Adoption: LIB-WIRED importers=3 nonval=1 — fully adopted (top of its ladder). | ● | ◉ | ◉ | ◉ | ○ |
Signed daemon registry with health-probed respawnMeasured:ds_verify_reg exists in runtime/_hdl_build/nx_daemon_supervisor.nx, verified at emit. A daemon is a REGISTRY ROW read hot every 15-second cycle, each probed by a health probe that distinguishes SERVING from HUNG from REFUSED and so catches the wedge a pidfile-or-port check misses, and the registry is signature-verified so a tampered file fail-closed holds last-good. THE MEASURED DANGER LIVES IN THE SAME ORGAN: its wedge-kill is what converted the 8-minute blackout from slow into unrecoverable, because every kill discarded a partially-completed startup and re-queued a fresh instance behind the same lock. Kubernetes has exactly one kubelet per node with restart authority and its startup probe exists precisely so a slow start is not read as death [k8s-probes]. Nothing here refuses a second supervisor, and two of them owned nx_tools_api at once during that incident. Adoption: LIVE-DAEMON — fully adopted (top of its ladder). | ● | ● | ● | ● | ● |
Scheduled-job heartbeat with an expected period and a grace windowMeasured:cw_row_declares exists in runtime/_hdl_build/nx_cron_watch.nx, verified at emit. The dead-man's-switch model -- a job pings on its own schedule and the watch reports STALE when the ping does not arrive inside period plus grace -- carried here only because the deploy, promote and rollback beats are traffic-safety machinery. Live at 17 watches and 0 stale as of this session. Coded zero across the field because none of the four columns watches cron; the monitoring-product comparison belongs to /compare/supervisor and is repointed, not restated. Adoption: RUN-BY:cron — fully adopted (top of its ladder). | ● | ○ | ○ | ○ | ○ |
Request-path admission and load sheddingOpen — watchingruntime/nx_ioadmit_lib.nx : ioa_request_admit, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. REPOINT NOT DUPLICATE, and the distinction is the whole row: /compare/loadgov owns HOST admission where ba_verdict and ioa_spawn_budget decide whether a compiler may FORK, on a D-state witness. NOTHING in the estate admits or sheds a REQUEST. Envoy is Best with five circuit-breaker threshold classes plus a retry budget expressed as a PERCENTAGE of live traffic, so retry load scales with success rather than with failure where a per-request cap of N multiplies under total outage [envoy-retry-budget], and its outlier ejection is bounded by max_ejection_percent so a global fault cannot make the detector eject the whole fleet [envoy-outlier-detection]. The contract deliberately extends the incumbent shared ruler rather than adding a second one. | ○ | ● | ● | ◉ | ○ |
Zero dropped connections across a process swapMeasured:ts_handoff_nodrop exists in runtime/nx_http_server.nx, verified at emit. THE INVARIANT, stated so it can be tested: there must be no instant at which zero processes hold the listening socket. THE ESTATE ALREADY DECLARED THIS RUNG AND ITS ACCEPT RULE AND HAS NEVER MET IT -- nx_pm_sclass_deploy carries the line A3 FD-passing zero-downtime hot-swap SCM_RIGHTS with the gate written as zero dropped connections across the swap, status ASSIGNED. HAProxy measured the unfixed version at 155 connection failures per million connections after 180 reloads and then removed the race entirely by MOVING the descriptor rather than re-binding it [haproxy-seamless-reload]. LANDED 2026-08-21 AND MEASURED IN DROPPED CONNECTIONS, NOT IN BINDERS. The blocker named here was real and is now gone: sys_sendmsg was ABSENT-PROVEN from all of buildroot/runtime at coverage_complete=1 and corpus_complete=1, and the shim now carries sendmsg, recvmsg and an SCM_RIGHTS send_fd and recv_fd whose struct offsets were read out of the platform headers rather than recalled. nx_scm_rights_gate proves a REAL listening socket moves between two REAL processes and keeps working after the original owner closes its copy, 14 of 14. nx_ts_handoff_gate then runs the accept rule itself in the socket-activation form, where an owner outside the service holds the listener and hands it to each generation so queued arrivals survive the restart [systemd-socket-activation]: ARMED 17 attempted, 17 served, ZERO drops, with the in-gap arrival connecting cleanly; NEG-CONTROL on the current bind-per-process path 17 attempted, 1 drop, the in-gap arrival refused with ECONNREFUSED. SCOPE STATED: that is the STRUCTURAL property, not a failure RATE -- no run of this size could see HAProxy's 155-per-million figure, which needs a load generator and a different instrument. TWO PATHS REACH THIS INVARIANT AND THE CONDITION THAT SELECTS BETWEEN THEM IS RECORDED HERE RATHER THAN ONE BEING TREATED AS SUPERSEDED. (a) ORDINARY FD INHERITANCE ACROSS execve, which is what systemd's LISTEN_FDS/LISTEN_PID contract is: the owner holds the bind and the generation inherits it, needing only fork, exec, dup2 and getenv -- every one of which this shim already had, so this path never needed sys_sendmsg at all. It satisfies the invariant BY CONSTRUCTION rather than by protocol, because the listener was never the process's to declare. (b) SCM_RIGHTS DESCRIPTOR PASSING, which is what nginx and HAProxy use and what shipped here. THE SELECTOR IS WHETHER THE SOCKET OWNER IS THE GENERATION'S PARENT: inheritance works only for a child the owner itself forked, and descriptor-passing is required for a peer it did not. OUR PRODUCTION CASE IS THE PEER CASE, measured from daemons.reg rather than assumed -- nx_daemon_supervisor and nx_hostctl fork the serving daemons, so a socket owner that is not the supervisor can never be their parent, and making it their parent would mean replacing the supervisor. BOTH PATHS ARE EXERCISED IN THIS ESTATE TODAY AND THE GATES ARE THE WITNESS: nx_ts_drain_gate's subject INHERITS its listener across a plain fork with no exec and no control message, while nx_survey_handoff_gate's adopted arm hands the descriptor across fork+EXECVE over SCM_RIGHTS. The inheritance path is cheaper and should be preferred wherever the owner forks the generation; it is not available to the adoption that shipped. Adoption: LIB-WIRED importers=64 nonval=52 — fully adopted (top of its ladder). | ● | ◉ | ◉ | ◉ | ○ |
Cooperative drain keep serving after TERM until a named deadlineMeasured:ts_drain_on_term exists in runtime/nx_ts_drain_lib.nx, verified at emit. LANDED AND ADOPTED 2026-08-21, and the ORGAN PATH ON THIS ROW WAS CORRECTED BY MEASUREMENT RATHER THAN LEFT TO ROT. It named runtime/nx_signal.nx, where this symbol structurally CANNOT work and where this watch could therefore never have flipped: TARGET_X86_64 is hard-pinned undefined so the x86 backend emits through x86ctx_rv64_to_x86_64_syscall, whose default branch is a pass-through, and rv64 134 is not a key in that table -- so nx_signal's own nx_signal_action_install lands on x86 uselib and cannot install anything on the live target. The drain therefore lives in nx_ts_drain_lib beside the TS1 handoff it is a sibling of. IT DOES NOT INSTALL A HANDLER AT ALL, which is the deeper point: a handler that RETURNS needs SA_RESTORER on x86-64, and a delivered signal interrupts a blocking syscall with EINTR -- a read or write cut mid-response IS the truncated reply this rung exists to prevent, so the mechanism that NOTICES TERM must not be the mechanism that can CORRUPT the response. TERM is BLOCKED and consumed as a pollable descriptor, which buys both properties at once and makes an idle daemon exit the instant TERM arrives instead of sleeping out its budget. THE DEADLINE IS DERIVED, NEVER TYPED: every accepted connection already carries sys_set_socket_timeout(cfd, ACCEPT_TMO_S), so one in-flight request cannot outlive ACCEPT_TMO_S by construction and the drain deadline IS that same named constant -- rung TS9's principle applied one rung early, with knowledge/trafficsafety_drain.conf shipped deliberately EMPTY of a pinned row so the linkage cannot be broken by a second edit. PROVEN IN WHAT THE CLIENT RECEIVED, not in what the daemon logged: nx_ts_drain_gate 11/11, ARMED complete_response=1 killed_by_signal=0 with the request PROVABLY in flight when TERM landed, NEG-CONTROL on the identical harness complete_response=0 killed_by_signal=15. Bite-proven both directions -- unblocking the signal takes it 11/11 GREEN to 6/11 RED with the armed arm collapsing to the neg-control's exact signature, and the restore is byte-identical. Kubernetes states the race precisely: endpoint removal happens AT THE SAME TIME AS shutdown begins and not before it, so the only correct implementation is to keep serving after TERM and finish open connections [k8s-pod-termination]. AWS names the client-visible cost of getting it wrong -- a target that terminates the connection before the deregistration delay elapses makes the client receive a 500-level error, and the default delay is 300 seconds with early completion when the target is idle [alb-deregistration-delay]. MEASURED HERE FULL POPULATION at corpus_complete=1: rt_sigaction resolves 8 times across buildroot/runtime and every one is SIGPIPE handling, a crash handler, or the nx_signal library itself -- ZERO are a TERM drain in a serving daemon. The primitive exists and the adoption is nil. Envoy is honest about its own limit, implementing graceful draining for only four filter types so a plain TCP proxy hard-closes [envoy-draining]. Adoption: LIB-WIRED importers=3 nonval=2 — fully adopted (top of its ladder). | ● | ◉ | ◉ | ◉ | ● |
Blue-green behind a front door that never restartsMeasured:ts_slot_flip exists in runtime/nx_ts_slot_lib.nx, verified at emit. LANDED 2026-08-21 AS A STANDALONE ROUTER, AND THE ORGAN PATH ON THIS ROW WAS MOVED OFF nx_proxy_route DELIBERATELY. That organ compiles into sites.elf, the front door all 69 route rows cross, so editing it to gain a deploy-safety property is the one change whose blast radius is every request in the estate -- the rung ships instead as nx_ts_slot_lib plus nx_ts_slot_router, fronting only throwaway slots, and adopting the pattern for the real front door is an operator decision to be taken AGAINST this evidence rather than smuggled in beneath it. ALL THREE CONJUNCTS OF THE ACCEPT RULE ARE TESTED SEPARATELY because a compound assertion that will not name its failing conjunct is a false-alarm generator: nx_ts_slot_router_gate 13/13 measured 48 attempted and 0 failed across a live flip, with 16 of 16 answered by slot A before it and 16 of 16 by slot B after; the front door was never restarted and was still the same live pid at the end; and a flip onto a slot whose server had been KILLED FOR REAL was REFUSED BY NAME with the active slot unmoved and 16 of 16 requests still reaching the healthy slot. The upstream choice is DATA re-read PER CONNECTION, so a flip takes effect on the next request with no signal, no restart and no reload race. IT INHERITS TS1 AND TS2 RATHER THAN RESTATING THEM: the front door takes its own listener from ts_handoff_nodrop, which is what makes the one component that must never restart itself replaceable, and it drains on TERM. Bite-proven -- deleting the warm-up guard takes it 13/13 GREEN to 8/13 RED and the mutant moves live traffic onto the dead slot, failing ALL 16 client requests, which is the outage the guard exists to prevent measured in what the client saw. SCOPE OF THE PROBE STATED: it proves the candidate accepts a connection and answers bytes, which is liveness and not health; the deeper instrument is nx_http_probe_lib hp_run with its thirteen distinct exit codes and the honest upgrade is to compose it, never to grow a second probe here. REPOINT: /compare/deploy rung DL1 owns blue-green WARM SLOTS as an artifact-pipeline contract and this row does not restate it. What is owed HERE is the property that makes it safe on one box -- the front door is the ONLY thing that must never restart during a deploy and the ONLY thing that must be independent of the service being replaced. Today the front door is sites.elf, a single process that all 69 live route rows cross, and it holds an ordinary listener with no hot path and no drain. Argo and Flagger are Best but both presuppose a traffic router and at least two replicas, so the products do not transfer even though the pattern does [flagger-strategies]. Adoption: LIB-WIRED importers=2 nonval=1 — fully adopted (top of its ladder). | ● | ● | ● | ◉ | ◉ |
Mirror to a shadow port and discard the responseOpen — watchingruntime/_hdl_build/nx_proxy_route.nx : ts_mirror_shadow, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. The zero-blast-radius canary and the cheapest real one: Flagger copies each incoming request to primary and canary, returns the PRIMARY response to the user and DISCARDS the canary response [flagger-strategies]. On a single box this is a shadow port and a diff and it needs no traffic router and no second replica at all, which is exactly why it transfers when weighted shifting does not. It also composes with the capability-loss oracle above -- a mirrored request answered differently by the candidate is the strongest available evidence before anyone depends on it. | ○ | ○ | ○ | ◉ | ◉ |
Bake with a minimum request denominator before promotingOpen — watchingruntime/_hdl_build/nx_mgmt_api.nx : ts_bake_window, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. Amazon's pipelines hold a stage for a bake window and wait for a specific number of data points, for example at least 100 requests to a named API, before advancing, precisely because a negative impact is often slow-burning and not readily apparent [aws-handsoff-deploy]. A rollback check that passes on zero requests is the gate-passes-on-the-empty-set defect wearing a deploy costume. Argo's third state is the honest one and is the shape to copy: an inconclusive analysis PAUSES at the current step rather than being folded into pass or fail [argo-rollouts-analysis]. | ○ | ○ | ○ | ○ | ◉ |
Tell the peer which requests were processed before closingOpen — watchingruntime/nx_http_server.nx : ts_goaway_lastid, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. RFC 9113 GOAWAY carries the last stream identifier the sender might have acted on, which is the only mechanism in the whole field that makes retry CORRECTNESS decidable rather than guessed, and it prescribes two frames with at least a round-trip between them so a connection can be cleanly shut down without losing requests; a connection that terminates WITHOUT a GOAWAY forces the client to treat the last stream identifier as the highest possible, so nothing non-idempotent can be safely retried [rfc9113-goaway]. HONEST SCOPE STATED RATHER THAN PROMISED: the guarantee exists only on HTTP/2 and above and our edge serves HTTP/1.1, which has Connection close and no last-processed marker, so this rung is bounded by the protocol and its cost includes that. | ○ | ● | ● | ◉ | ○ |
Wait hints derived from the real recovery boundOpen — watchingruntime/_hdl_build/nx_mgmt_api.nx : ts_retry_after_derived, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. RFC 9110 defines Retry-After as an HTTP-date or a number of seconds with NO range, NO jitter and no accuracy obligation, so a scalar synchronises the herd by construction and a wrong value is fully spec-compliant [rfc9110-retry-after]. OUR MEASURED INSTANCE: an advertised retry-after of 30 seconds against a real 300-second lock TTL, so an honest caller obeying the advice hammered a lock that could not possibly be free, and a census found four sibling lanes with the same mismatch, three of them twenty times short. THE CORRECTION IS THE QUESTION AND NOT THE NUMBER -- a 300-second exclusive lock is not a retry situation at all, so the right answer is a non-retryable error class, and any hint that is emitted must be derived from the same named constant as the lock rather than typed a second time. | ○ | ● | ● | ● | ○ |
The undo path is proven not to traverse the subjectOpen — watchingruntime/_hdl_build/nx_mgmt_api.nx : ts_undo_independence, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. THE CENTRAL AXIS OF THIS DOMAIN, and the field's position is doctrine everywhere and a tested property NOWHERE. AWS states it four ways -- the data plane keeps working under control-plane impairment, recovery takes no deliberate control-plane dependency [aws-static-stability], use data-plane API operations with pre-stored credentials for recovery, and monitor from somewhere that does not share fate [arc-dataplane-recovery] -- and then its own post-event summary concedes that the situation had no established operational recovery procedure [aws-us-east-1-2025-pes]. MEASURED FOR US FROM THE LIVE ROUTE TABLE RATHER THAN ASSERTED: /api resolves to the mgmt daemon and /api/tools and /mcp to the tools API, and BOTH are reached through the single sites.elf front door, so the process whose restart is the incident sits on the path to the call that undoes it -- with an 8-minute estate-wide blackout already on record. THE CHEAPEST FIX IS NOT A SECOND CHANNEL, IT IS A PROCESS YOU DID NOT KILL: nginx rolls back by signalling a master that never stopped running, touching no build and no promote [nginx-binary-upgrade], while ours re-promotes a banked binary and therefore needs the build path alive at exactly the moment it may not be. A gate that resolves the dependency chain of the rollback ACTION and REFUSES when it passes through the thing being rolled back is the enforcement the field only writes essays about, and it is the second exceed this domain can earn. | ○ | ○ | ○ | ○ | ○ |
Automated rollback triggered on capability lossOpen — watchingruntime/_hdl_build/nx_mgmt_api.nx : ts_rollback_on_caploss, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. The oracle exists and the TRIGGER does not: rg_missing and rd_in_baseline already answer whether the new build still does everything the old one did, and nothing fires a rollback from them. Argo is Best at the mechanism, with analysis results driving the state machine and a failed analysis setting the canary weight back to zero [argo-rollouts-analysis]. THE COUNTERWEIGHT IS PART OF THE CONTRACT AND NOT A CAVEAT: in the October 2025 us-east-1 event health checks alternating between failing and healthy drove automatic failover that took HEALTHY capacity out of service, the alternation increased load on the health-check subsystem until it degraded, and the mitigation that worked was DISABLING the automation [aws-us-east-1-2025-pes]. So this rung ships only with all four of a two-window burn-rate signal whose short window stops the action latching after recovery [sre-slo-burn-alerts], a bounded number of automatic actions per unit time, a minimum-request denominator, and a time-boxed off switch that AUTO-RE-ARMS -- the last of which no progressive-delivery controller ships. | ○ | ○ | ○ | ○ | ◉ |
Risk register
| Risk | Likelihood x impact | Mitigation |
|---|---|---|
| Two overlapping listeners are exactly what the duplicate-port detector is built to flag, so TS1 makes a working guard fire on correct behaviour | likely x medium | TS1 must teach nx_portdup to tell a bounded handoff overlap from a squat in the SAME change that creates the overlap -- never by widening the detector to ignore the port, and never by shipping the overlap first and fixing the alarm later. |
| Automated rollback on a measured signal is the mechanism that caused the outage this domain cites, so TS6 can make things worse rather than better | possible x high | TS6 does not ship without all four safeties as separate teeth, and its gate must include a flapping-input fixture that FAILS the rung if the action rate is unbounded. If any of the four cannot be proven, the rung ships in report-only mode. |
| A drain deadline is a number somebody picks, which is the magic-number defect this estate refuses | likely x medium | TS2's deadline is derived from the measured distribution of in-flight request durations already observable on the serving path, published with its sample size, and refuses to arm below a declared minimum number of samples -- the same arming protocol the load-ceiling rung uses next door in loadgov. |
| The front door becomes a new single point of failure that is now also load-bearing for deploys | possible x high | TS3's front door does the smallest possible job -- choose an upstream -- and TS1 must land first so the front door itself is replaceable without a gap; a front door that cannot be replaced hitlessly is refused rather than deployed. |
rg_missing names every route the live binary serves and the candidate does not, and rd_in_baseline catches any route vanishing across a deploy -- and both were built after the regression happened to us, five measured times. The weak side is not a gap we can wave away, and the operator's instinct was right. We replace the single serving process every caller uses, with no drain, no overlap on the front door, no traffic shifting and no automated rollback. Three numbers say it precisely, each full-population with declared coverage. First: nx_http_server_listen_hot resolves 9 times across the whole tree and exactly 2 are production call sites -- nx_mgmt_api and nx_tools_api_serve -- so the sites.elf front door that all 69 live route rows cross holds an ordinary listener and cannot hand off at all. Second: rt_sigaction resolves 8 times and every one is SIGPIPE handling, a crash handler, or the nx_signal library itself -- zero were a TERM drain in a serving daemon when this page was admitted, and that is now ONE -- nx_survey_daemon arms ts_drain_on_term before its listener exists. And the census pointed at a file that could never have carried it: rt_sigaction is spelled RV64 134 in nx_signal.nx, 134 is not a key in x86ctx_rv64_to_x86_64_syscall, and that table's default branch is a pass-through -- so on the live x86 target nx_signal_action_install lands on uselib and installs nothing. The 8 references were never the adoption gap they looked like; the shipped drain therefore installs no handler at all, blocking TERM and consuming it as a pollable descriptor, which also removes the EINTR mid-response truncation a handler would have introduced, and Kubernetes names the precise consequence: endpoint removal happens at the same time as shutdown begins, so a process that exits on TERM is what produces the client-visible 5xx [@k8s-pod-termination]. Third: sys_sendmsg was ABSENT-PROVEN from all of buildroot/runtime at coverage_complete=1 and corpus_complete=1, so the mechanism the field actually uses -- passing the listening descriptor over SCM_RIGHTS [@haproxy-seamless-reload] -- could not be written at all. That gap was closed on 2026-08-21 and this sentence is corrected rather than left standing: the shim now carries sendmsg, recvmsg and sys_send_fd/sys_recv_fd, and ts_handoff_nodrop in nx_http_server is proven against the pre-declared accept rule in DROPPED CONNECTIONS: the handoff arm drove 17 connections across a real process swap for zero drops with the arrival during the gap connecting cleanly, while the identical harness against today's bind-per-process path lost one and refused the in-gap arrival with ECONNREFUSED. The remaining deficit WAS adoption, and that half closed on 2026-08-21 -- this sentence is corrected rather than left standing, because a stale claim is worse than the original error. nx_survey_daemon is the FIRST ADOPTER: it calls ts_handoff_nodrop instead of binding, and the adoption is proven in DROPPED CONNECTIONS across a real fork+exec restart of the real binary, not by the source referencing a symbol -- nx_survey_handoff_gate 12/12, ADOPTED arm attempted=17 served=17 drops=0 with the in-gap arrival connecting cleanly, NEG-CONTROL the same binary with no owner attempted=17 served=16 drops=1 and the in-gap arrival refused ECONNREFUSED. The subject was chosen from the route table rather than from taste: it carries exactly ONE of the 69 live rows, owns its binary outright, and fronts no control-plane path -- deliberately NOT sites.elf, which is rung TS3. Envoy's ordering rule went in with it: its three arenas and its seg-store warm now run BEFORE the socket is acquired, so the publish returning is itself evidence that initialisation completed while the outgoing generation was still serving. What remains is one deploy decision and one owner process, not a capability gap -- and because ts_handoff_nodrop falls back to binding when no owner publishes, the adopted daemon is byte-for-byte safe to deploy before any owner exists. And the primitive we did adopt is the weaker one. LWN documents SO_REUSEPORT as carrying a defect: when the number of listening sockets bound to a port changes, connections can be dropped during the three-way handshake, because the final ACK may not be routed to the socket that received the SYN [@so-reuseport-lwn]. So our own nx_hotlisten_gate proves the necessary condition -- two processes can hold the port -- and never the sufficient one, zero drops. The estate had already written the right accept rule and never met it: nx_pm_sclass_deploy carries A3 FD-passing zero-downtime hot-swap GATE=0-dropped-conns-across-swap status=ASSIGNED. The central axis is the undo path. Static stability is doctrine stated four ways by one organisation -- the data plane keeps working under control-plane impairment, recovery takes no deliberate control-plane dependency, use data-plane operations with pre-stored credentials, monitor from somewhere that does not share fate [@aws-static-stability] [@arc-dataplane-recovery] -- and it is a tested property nowhere; AWS's own post-event summary concedes the situation had no established operational recovery procedure [@aws-us-east-1-2025-pes]. Read from our live route table, /api resolves to the mgmt daemon and /api/tools and /mcp to the tools API, and both are reached through the one sites.elf front door -- so the process whose restart is the incident sits on the path to the call that undoes it, and an 8-minute estate-wide blackout is already on record. The cheapest fix is not a second channel, it is a process you did not kill: nginx rolls back by signalling a master that never stopped running, touching no build and no promote [@nginx-binary-upgrade], while ours re-promotes a banked binary and therefore needs the build path alive exactly when it may not be. And the counterweight is not optional. Automated rollback on a measured signal caused the October 2025 outage: health checks alternating between failing and healthy drove automatic failover that took healthy capacity out of service, the alternation degraded the health subsystem itself, and the mitigation that worked was disabling the automation [@aws-us-east-1-2025-pes]. Anything we build here ships with a two-window burn-rate signal [@sre-slo-burn-alerts], a bounded action rate, a minimum-request denominator, and a time-boxed off switch that auto-re-arms -- which no progressive-delivery controller ships. One more honest number. Of the routing and edge organs censused, 6 of 8 are REGISTERED-DARK -- callable, authorised, with no invocation on record -- and the two daemons that ARE the traffic path, nx_hub_gw and nx_daemon_supervisor, are PROMOTED-UNREGISTERED, real binaries nobody can call. The safety tooling is not missing. It is dark.Person · product · place — not yet measured for this domain
knowledge/compare/trafficsafety.ppp (rows surface|nishi or c1..c4|label|url|connect naming OUR live surface and each rival's front door), run nx_ppp_probe domain trafficsafety, and this section fills itself on the next beat: the same ruler on both sides — privacy and CX (third-party hosts, tracker classes, cookies, security headers), design and longevity (design hygiene, computed WCAG contrast, render-blocking resources, unsized media, script weight, theme and motion queries), findability (landmarks, skip link, on-site search, breadcrumb, headings, internal links).References
- [so-reuseport-lwn] Kerrisk, M. The SO_REUSEPORT socket option. LWN.net, March 2013. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_so-reuseport-lwn.html· pinh1b28cf47fc0cc10ee274942642444e32837f6aa0d1fbc4f647f2b23a69b997ae· accessed 2026-08-21 · mirrored-indexGrounds: Hot listener a second process can hold the port while the old still serves -- the mirror carries verbatim that a change in the number of listening sockets can drop connections during the three-way handshake, and calls it a defect in the implementation - [haproxy-seamless-reload] HAProxy Technologies. Truly Seamless Reloads With HAProxy -- No More Hacks. HAProxy blog. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_haproxy-seamless-reload.html· pinh81249aec4560ba23430c269a25932729fd83596b618ee9edd726db8171c5881f· accessed 2026-08-21 · vendor-docGrounds: Zero dropped connections across a process swap -- descriptor transfer over SCM_RIGHTS and the measured 155 connection failures per million before the fix - [nginx-binary-upgrade] nginx documentation. Controlling nginx -- master and worker signals and upgrading the executable on the fly. Archive snapshot read 2026-08-21 because nginx.org fails our TLS client. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_nginx-binary-upgrade.html· pinh2dc954fc22c9607d7201f13eca031d2b3be6c77d92ec880fc879021d1ea4150d· accessed 2026-08-21 · vendor-docGrounds: The undo path is proven not to traverse the subject -- rollback is a HUP to a master process that never stopped running, touching no build and no promote - [envoy-hot-restart] Envoy Proxy architecture overview. Hot restart -- process handoff order, socket copying, drain and parent-shutdown timers. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_envoy-hot-restart.html· pinh41536cc52e01428d8a783ccd090c743ecfd6b3a8e60497f1705d3e03a0334e39· accessed 2026-08-21 · vendor-docGrounds: Hot listener a second process can hold the port while the old still serves -- the new process fully initializes itself BEFORE it asks for copies of the listen sockets, and existing connections are never transferred - [envoy-draining] Envoy Proxy architecture overview. Draining -- drain triggers, graceful versus immediate, per-filter drain support. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_envoy-draining.html· pinh78ca35517a7323f3754b497b7330b84198abd17cf0a2ed64f9bdf1f242407931· accessed 2026-08-21 · vendor-docGrounds: Cooperative drain keep serving after TERM until a named deadline -- graceful draining is implemented for only four filter types and a plain TCP proxy hard-closes - [envoy-outlier-detection] Envoy Proxy architecture overview. Outlier detection -- passive health checking, ejection algorithms and the max ejection cap. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_envoy-outlier-detection.html· pinh818560700088c8dbb774be2b9f047d6624e57cb0fc273b6469265c7fa6a6dd09· accessed 2026-08-21 · vendor-docGrounds: Request-path admission and shedding -- max_ejection_percent is the bounded-remedy pattern so a global fault cannot make the detector eject the whole fleet - [envoy-retry-budget] Envoy v3 API. CircuitBreakers Thresholds RetryBudget -- budget_percent, min_retry_concurrency, budget_interval. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_envoy-retry-budget.html· pinhaa7f1e4da0995d4b385f897ca740dc7336bd33520ed659ae8ba691d47c656245· accessed 2026-08-21 · vendor-docGrounds: Request-path admission and shedding -- retries capped as a percentage of live traffic so retry load scales with success rather than with failure - [systemd-socket-activation] systemd.socket manual page section 5. man7.org mirror of the systemd manual. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_systemd-socket-activation.html· pinh2a5afec02e7666f911244e888dd1700b45e0d2c36259054b9292b144e21bff1f· accessed 2026-08-21 · vendor-docGrounds: Zero dropped connections across a process swap -- the init system owns the listener and passes the descriptor in, and FlushPending=no keeps queued connections across a restart - [k8s-pod-termination] Kubernetes documentation. Pod Lifecycle, Termination of Pods. Read from the website repository source so the ordering is exact. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_k8s-pod-termination.md· pinha22f3a96a41e7613e61beb292fc28fa45b9f68e8f65e493e3e940ed1d35a8b51· accessed 2026-08-21 · source-readGrounds: Cooperative drain keep serving after TERM until a named deadline -- endpoint removal happens AT THE SAME TIME AS shutdown begins, not before it, so the process must keep serving after TERM - [k8s-probes] Kubernetes documentation. Liveness, Readiness and Startup Probes -- what each probe does on failure and the cascading-failure caution. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_k8s-probes.html· pinh827ab5770d901c14477749e9d9377b60f847185349c29c73ceaa46d5fedf05fd· accessed 2026-08-21 · vendor-docGrounds: Signed daemon registry with health-probed respawn -- the startup probe exists so a slow start is not read as death, and incorrect liveness probes are named as a cause of cascading failures - [alb-deregistration-delay] AWS Application Load Balancer target group attributes -- deregistration delay and slow start mode. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_alb-deregistration-delay.html· pinh642a59d4e4aea00df7e981abbcb082436b4fe3c9d6d8cfa0f8fb3e0d6dff3c0e· accessed 2026-08-21 · vendor-docGrounds: Cooperative drain keep serving after TERM until a named deadline -- a target that terminates the connection before the deregistration delay elapses makes the client receive a 500-level error - [argo-rollouts-analysis] Argo Rollouts documentation. Analysis and Progressive Delivery. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_argo-rollouts-analysis.html· pinh67194959b0d3f3b1d7adf4deec2c1acbc05017d4f90b46d81906d30dd54f378d· accessed 2026-08-21 · vendor-docGrounds: Bake with a minimum request denominator before promoting -- an inconclusive analysis PAUSES at the current step rather than being folded into pass or fail - [flagger-strategies] Flagger deployment strategies documentation. fluxcd/flagger repository main branch. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_flagger-strategies.html· pinh031da19b164ef1b2bed28a8007127c14047703fce103087c8e4084767c1ce015· accessed 2026-08-21 · source-readGrounds: Mirror to a shadow port and discard the response -- traffic mirroring sends the primary response to the user and DISCARDS the canary response, the zero-blast-radius canary - [cas-canary-analysis] Davidovic with Beyer. Canary Analysis Service. ACM Queue volume 16 issue 1, 2018. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_cas-canary-analysis.html· pinhb8e278e9ce33aad8e03e7c6962a23cec4d6d8c218330858e1a64c78e839452fe· accessed 2026-08-21 · published-paperGrounds: Promote refused when the candidate serves fewer routes than live -- CAS asks whether the canary is meaningfully worse and states outright that it is NOT trying to establish absolute health - [aws-static-stability] Becker and MacCarthaigh. Static stability using Availability Zones. Amazon Builders Library. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_aws-static-stability.html· pinh8ed40f771c551b12e7abb8594753e9dbbbfa0cdcb5a39d5e9a23798f04a3d2d0· accessed 2026-08-21 · vendor-docGrounds: The undo path is proven not to traverse the subject -- the data plane keeps working under control-plane impairment and recovery takes no deliberate control-plane dependency - [arc-dataplane-recovery] AWS Application Recovery Controller. Best practices for zonal shifts. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_arc-dataplane-recovery.html· pinh46a501e69d86d83f100d4542c608ce03d375c10784e1fbee5647d8e5bceab15c· accessed 2026-08-21 · vendor-docGrounds: Reachability measured from a second vantage that does not share fate -- monitor from a second adjacent Region so application and monitoring are not impaired by the same event, and use data plane operations with pre-stored credentials for recovery - [aws-us-east-1-2025-pes] Summary of the Amazon DynamoDB service disruption in the Northern Virginia US-EAST-1 Region. AWS post-event summary, October 2025. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_aws-us-east-1-2025-pes.html· pinhf1c181cf8f5bf0b82ba3d55d2f222ea341bb94f573fee9759674db3f701b2a40· accessed 2026-08-21 · vendor-docGrounds: Automated rollback triggered on capability loss -- the COUNTERWEIGHT: flapping health checks drove automatic failover that removed healthy capacity, and the mitigation that worked was disabling the automation - [aws-handsoff-deploy] Liguori. Automating safe hands-off deployments. Amazon Builders Library. Archive snapshot read 2026-08-21 because the canonical URL redirects to a shell page. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_aws-handsoff-deploy.html· pinhe2fbb62ed6af7f3259fed432780abb4608da1a1701b5d3df317bd4325684a862· accessed 2026-08-21 · vendor-docGrounds: Bake with a minimum request denominator before promoting -- bake time waits for a specific number of data points, for example at least 100 requests, and a one-box stage carries its own scoped rollback alarm - [sre-slo-burn-alerts] Google Site Reliability Workbook chapter 5. Alerting on SLOs. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_sre-slo-burn-alerts.html· pinh5a2195ae9280d92489ec6c5be9781ef191d73137514da7c716ebabf5479cfcf5· accessed 2026-08-21 · published-courseGrounds: Automated rollback triggered on capability loss -- the signal shape is a two-window burn rate where the short window exists so the action does not latch after recovery - [rfc9113-goaway] Thomson and Benfield. HTTP/2. IETF RFC 9113, section 6.8 GOAWAY. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_rfc9113-goaway.txt· pinha00ef91b64e111a282e77ec66980f5242e77c0bb5e33e0927e3b6757080506de· accessed 2026-08-21 · published-standardGrounds: Tell the peer which requests were processed before closing -- the last stream identifier makes retry correctness decidable, and a connection closed without GOAWAY forces the client to assume everything may have been processed - [rfc9110-retry-after] Fielding, Nottingham and Reschke. HTTP Semantics. IETF RFC 9110, section 10.2.3 Retry-After. publisher · read in our library
knowledge/fetched/cmp_trafficsafety_rfc9110-retry-after.txt· pinh21c1cdce6ab0e5509b04d84a28000836c7a087cf786efe6f04877ebfff47232a· accessed 2026-08-21 · published-standardGrounds: Wait hints derived from the real recovery bound -- Retry-After is a bare scalar with no range and no jitter and the spec imposes no accuracy obligation
generated by nx_swcompare_matrix (sovereign NishiLang organ) from knowledge/compare/trafficsafety.matrix · every Nishi cell verified against organ source at emit time · watch cells re-measured on every compare beat · zero JS, zero trackers