Nishi Family › Compare › External Model Workers
Nishi Compare · capabilities, evidence and delivery
External Model Workers
Compare capabilities and delivery evidence. Source discovery, qualification, deployment and accepted outcomes are separate; an open contract names work still to verify.
Nishi classified-egress worker lane (free inference as a U-cleared sink, byte-exact judge, egress ledger, terms as data) vs LiteLLM gateway and the OpenRouter router and LLM Guard egress scanning and openai-evals model grading -- Aug-2026 field
Overview
The purpose, declared position and evidence coverage of this domain. Source presence and completed acceptance are different measures.
Where we are. MEASURED 2026-08-24 against the catalogue and the mirrored terms: the sovereign TLS 1.3 client and a JSON POST over it are LIVE (yt_post, nx_https_req_complete), the vault and its lease rotation are BUILT (vlt_derive, rot_due), the clearance resolver and the public U index are LIVE (ag_uid_to_level, dss_search_off_div), the evidence register and its pin gate are LIVE (rf_provenance, crg_pin_ok), the byte-exact judge and the forge prompt and fence machinery are LIVE (fg_run, fg_bank, fm_compose_prompt, fm_strip_fences, fc_emit), the plane ledger verb is LIVE (nsp_hist_append) and the SWE-bench Verified grader is BUILT (sg_verdicts). CORRECTED 2026-08-25, MEASURED: XL1 and XL2 ARE NOW LIVE. nx_extllm (nx_catalog VERDICT LIVE -- source, built, promoted, registered, authorised, invoked, with_gaps=0) reads the 13-provider terms ledger and the release manifest, both of which existed as DATA since 2026-08-24 with ZERO readers in the whole 23,235-file corpus. Its gate is 27/27 GREEN on the live NAS binary and BITE-PROVEN twice: mutating the cumulative bound to a per-call check killed the aggregation tooth (RED 24/25) while its fixture-reached-the-condition partner stayed PASS, and mutating the classifier fail-open killed exactly the undeclared-source and empty-manifest neg-controls while the stale-hash tooth stayed PASS. Promote rulers agreed: nx_contentdiff lost_from_live=0 over 136 runs, nx_behaveprobe first_diff_off=-1 IDENTICAL STRONG. An UNPINNED terms row now ADMITS NOTHING, so a provider posture we have not pinned to a mirrored contract cannot be asserted. The Claude tier is in the ledger keyed on ACCOUNT TIER, not vendor name, because Free/Pro/Max and API/Team/Enterprise publish OPPOSITE defaults; all four rows are pinned to mirrors fetched 2026-08-25, and the UNDECLARED row REFUSES until the operator declares which tier this estate runs under. DISTILL is refused for anthropic on every tier -- both contracts forbid using outputs to train a competing model and our no-float LM is exactly that -- so the sovereign corpus can never be grown from Claude output. STILL ABSENT: NO external call organ, NO pool scheduler, NO external forge backend, NO best-of-N, NO LLM judge, NO egress ledger PLANE (the odometer arithmetic ships and is gate-proven, but nothing writes a row per call yet) and NO publish guard. The subject provider census (27 pages mirrored, 179 fetches by the research lane) is summarised in the Market row below.
Where we need to go. OPERATOR ASK (2026-08-24): stand up the OpenRouter stealth key as a capability, publish a /compare on it, and research the August-2026 opportunities to farm training and other work off free resources where the material is already public -- with a classification system like the estate's top-secret-down model applied to every such use, and with the free models working the open code-emitting rungs toward the /compare/coding and /compare/lang bars while the gates rule and the assistant only judges. Deliverable: the classified call lane (terms ledger, egress classifier, call plus scheduler plus ledger), the external forge tier (public pack, backend, best-of-N), the judge panel, the rung worker, the publish guard and the terms-gated distillation harvest.
Research bar. Stealth Program EULA, the contract that binds a free stealth model is measured on OpenRouter, updated 2026-07-06. Theirs: models are offered free of charge for purposes of collecting User Content for use in Stealth Model training and improvement; an irrevocable perpetual licence to the provider; a ban on publicly disseminating confidential technical information regarding the performance of the models [stealtheula]. Ours: XL1 carries these as rows and XL8 enforces the publish ban; the model page banner that says the opposite [oxalpha] is recorded beside it as the marketing statement it is.
Research bar. Large Language Monkeys, sampling against a verifier is measured on arXiv 2407.21787. Theirs: coverage climbs with the number of samples when a verifier exists [llmonkeys]. Ours: XL5 -- the estate's verifier is a byte-exact equality and a free sampler makes N affordable.
Research bar. Groq Services Agreement, the free tier that does not train and leaves outputs yours is measured on Groq, effective 2026-06-22. Theirs: Groq is not permitted to use Inputs or Outputs for training or fine-tuning any models unless explicitly granted permission; the customer retains all rights in Inputs and Outputs [groqsa]; free tier limits published per model [groqlimits]. Ours: XL9 -- the first provider whose outputs may lawfully grow the forge ledger; XL3 schedules to its published caps.
Business case
| Axis | Position | Figures | Decision |
|---|---|---|---|
| Investment | Zero token cost on every free tier in the census; the whole investment is organ work (12 u to version 2.0) and the operator's release decision for the manifest; no hardware and no credit card on the Groq, OpenRouter free, Cloudflare and Z.ai paths [groqlimits] [orlimits] [cfpricing] [zaitos], a card on Cerebras [cerebrastos] | 12 u; the return is rungs climbed on other boards by workers that cost nothing per token | fund XL1 to XL3 now -- the terms ledger and the classifier are reused by every external call the estate will ever make |
| Market | The August-2026 free-inference field from the mirrored record: OpenRouter routes one stealth model and a free collection where some providers train on prompts and OpenRouter itself does not [orfree] [orprivacy], with a per-request data_collection deny switch [orrouting] and published free-tier caps [orlimits]; Google's unpaid Gemini tier uses submissions to improve its products and forbids developing competing models [geminiterms] [geminipricing]; Groq's free tier does not train and leaves outputs yours [groqsa]; Cerebras replaced its free tier with a card-gated trial and forbids benchmarking [cerebrastos]; SambaNova's free tier does not train but forbids publishing performance results [sambanovaeula]; Mistral trains by default on preview models and on opt-in-by-default products [mistraltos]; Cloudflare gives a daily neuron allowance and does not train on customer content [cfterms] [cfpricing]; NVIDIA's API catalogue is trial-only, collects content to improve its models and forbids production use [nvidiatrial]; Z.ai's Flash models are free, not trained on, and their outputs may not train a competing model [zaitos]; DeepSeek is paid, stores data in the PRC and explicitly permits distillation [deepseektos]; Alibaba Model Studio gives a per-model free quota and states it never trains on customer data [alibabaquota] [alibabaprivacy]; GitHub Models was retired on 2026-07-30 [ghmodels] | three classes fall out and the ledger encodes them: WORKER (any provider, U material only), JUDGE (any provider, U artifacts), DISTILL (Groq, Cloudflare, DeepSeek only) | no provider is trusted from its model card; every row is read from the pinned contract |
| Opportunity cost | The forge today samples from a local 0.5B and 1.5B tier that pattern-copies; a free frontier-class sampler behind the same byte-exact judge converts the coding domain's best-of-N rung from a throughput problem into a scheduling problem | the ruler stays ours, the samples become free | charge XL1, XL2, XL3 and XL8 to the platform; XL4 to XL7 to the coding domain; XL9 to the modelwright domain |
| Competitive position | The gateway field owns routing and spend [litellm]; the DLP field owns pattern scanning [llmguard]; the evaluation field owns model-graded evals [openaievals]; none of them owns a clearance model, a terms ledger read from pinned contracts, or a byte-exact judge on the worker's output | our position is the guard and the ruler, not the model | use their inference, keep our classification, publish only what the contract permits |
| Buy vs build | Inference: TAKE FREE where the terms row permits the use, BUY DeepSeek for distillation if volume is wanted [deepseektos]. Transport, vault, classification, judge, ledger: already BUILT. Terms ledger, classifier, scheduler, backend, panel, worker, guard, harvest: BUILD -- none exist to buy in the clearance-aware form | the only build is the guard and the loop; the model is never ours to own on this lane | guard first, worker second, harvest last |
Market-entry ladder
| Level | Best in class (Aug 2026) | What we have | What must grow | Verdict |
|---|---|---|---|---|
| Hobbyist | Paste a prompt into a free model's page or a router's playground [oxalpha]; no clearance, no ledger, the prompt is the payment [stealtheula] | the classification rule as a written instruction: only what the public site already serves may be pasted | nothing structural -- the discipline of the operator and the assistant | ENTER NOW by hand on U material only; it is how the 21-call probe of 2026-08-24 was run, and it is where the manifest's first rows come from |
| Small business | A gateway with key management, retries and spend logs [litellm], a router with per-request data-collection routing [orrouting] and published caps [orlimits] | the sovereign transport, the vault and rotation, the plane ledger verb, the pin gate | XL1 the terms ledger, XL2 the classifier, XL3 the call with the scheduler and the ledger | GROW-FIRST -- enter when M0 is green; until then every external call is a hand-run probe on U material and nothing else |
| Market leader | Enterprise gateways with DLP scanning of prompts [llmguard], zero-data-retention contracts with the inference vendors [groqsa] [cfterms], model-graded evaluation suites [openaievals] | a clearance model the field does not have (ag_uid_to_level, the public U index with a consent bit) and a byte-exact judge the field cannot have | XL2 at the door, XL6 the panel, XL8 the guard | HOLD until M2 -- and the position is stronger than the leader's once XL2 ships, because a level is a rule where a pattern list is a floor |
| Innovation leader | Nobody in the field combines clearance-by-construction egress, provider terms as pinned data consumed at the call, and a deterministic verifier judging the worker's output; the closest published result is sampling against a verifier [llmonkeys] | the verifier and the classification are measured present; the wiring is the whole gap | XL4 and XL5 published on the coding board with the external tier as the sampler | THIS IS THE WEDGE -- reachable at M1 without any model of our own, which is why the entry order is guard, then worker, then harvest |
| Research leader | Distillation from frontier outputs where the licence permits it [deepseektos] [groqsa], repository-level agents graded on SWE-bench [swebench], contamination-aware evaluation | the SWE-bench Verified grader owning its denominator (sg_verdicts) and the compile-verified forge ledger | XL7 rung workers feeding the ledger and XL9 the harvest, then the modelwright domain's training arc over it | AFTER M3 -- the ledger is the research asset and it accrues from the first worker run, so the research rung is a by-product of the worker rung, not a separate investment |
Latest recorded release
2026-09-13 · XL5
2026-09-13 Canonical nx_forge_judge now has eval-release for fixture and response JSON argument values (not file paths). Native helper gate16/16; canonical CLI1460 passed escaped equivalent key and preserves unknown build evidence; usage1480 returns expected2. Structured fixture/response SHA256 receipts, no substring acceptance for this command, never release authority. Callable through existing mgmt native qualification; named route registration remains unqualified. No provider calls or ROI claim. Evidence buildroot/knowledge/gates/forge-evidence-adoption-t274.json.
Release entries describe recorded changes; they do not establish that every capability passed evaluation.
22 of 31 capability source symbols found|8 of them authored stronger-claims|9 open|coverage 709/1000|adoption 12 full / 10 partial
Evidence profile — what the gaps on this board actually are
Measured by nx_swcompare_evidence, read back by nx_evprofile_lib. Every figure is a count with its denominator — there is deliberately no score, no grade and no percentage anywhere in this band, because a stored scalar is a field a seat can edit and a counted partition is not.
evidence|grounded 21/21|unsupported 0|gates green 0/0|proven able to fail n/a — no gate ran|green at 0/0 0|open gaps 10|of them unnamed 0|of them proof withheld 0|flips ready 3
proven able to fail counts the gates that have a RECORDED RED — nx_gate_bite mutated the gate subject, rebuilt it, watched the gate go red, and that record is inside the shared TTL. never bitten is its complement over the same denominator: those gates ran and were green, and nothing has ever shown them able to detect anything, so their green is a statement about this run and not about the gate. green at 0/0 is a separate and much weaker observation — the gate printed GREEN on a zero denominator, so its own tooth counter says it examined nothing. A gate can be green, non-zero, and still never bitten; that is the common case and it is now visible instead of implied.
partition: grounded + unsupported = 21 vs present 21 · named + unnamed + withheld = 10 vs open 10 · both reconcile
liar-kill conj=GPqN · first failing conjunct: quorum
graded document: BUILDROOT tree, 14461 bytes · gates map: NONE · stamped 37d 6h ago · STALE against the shared TTL of 604800s — this is evidence about a build nobody can vouch for now · source ../knowledge/status/evstamp_extllm.verdict
| Gap class | What it is, and the work it names |
|---|---|
| CLAIM-ONLY | No declared gate ran for this board, so every capability mark here rests on a symbol existing in source and on nothing executing. The remedy is to WRITE a .gates map naming gates that already run — authoring, not research. |
| FLIP-READY | A watch contract symbol is now present in the organ it named: the cell can flip on a pure data edit with no new code. These are the cheapest closures on this board and they belong before any new work. |
| STALE | This profile is older than the shared TTL, or its clock ran backwards. It is reported as unusable rather than as fresh, because being wrong in the direction of refusing is the safe direction. |
nx_swcompare_evidence extllm itself and are not carried on the stamp, so this page names the classes and the producer names the rows. That split is stated rather than hidden: a count without a worklist is not actionable, and this band is honest about which half of that it is.Production map
Follow the dependencies, declared acceptance criteria and recorded priorities. Inspect source binding before treating a rank as executable work.
Ranking source binding: STALE_OR_INVALID_BINDING. Recorded priorities require current acceptance evidence and resource checks before execution.
Recorded ranking withheld
Retained ranking is not bound to the current plan and matrix: STALE_OR_INVALID_BINDING. Historical data remains retained. Refresh the ranking after resolving the input differences. Latest execution outcome is unverified.
Declared roadmap — contract, acceptance, executor, effort
| Rung | Closes with | Definition of done (pre-declared) | Executor | Est. |
|---|---|---|---|---|
| Provider terms ledger (XL1) | xl_terms_ledger | one row per provider read from the PINNED mirror, never the live site: trains_on_input, outputs_train_ours, benchmark_publish_restricted, retention, card_required, rate caps; UNKNOWN is a value and REFUSES every gated use; neg-control: a provider with no mirror row must never read ALLOW [ortos] [orprivacy] [orlimits] | Organ | 1 u |
| Egress classifier and release manifest (XL2) | xl_egress_class | every prompt byte resolves to a public-index cid or to a row of the operator-declared release manifest, else the call REFUSES naming the offending source; the manifest is data the operator writes, never a default; neg-control: a prompt assembled from an unreleased runtime .nx must never leave; positive control: a prompt built only from public compare pages must pass | Organ | 1.5 u |
| Call, scheduler and egress ledger (XL3) after XL1,XL2 | xl_call | request built from a conf row (provider, model, effort, max tokens) and sent through nx_https_req_complete with the key read from the vault; the 429 attributed to its source (shared pool versus this key) and the body state named; one lane per free pool with backoff derived from the measured refusal rate and concurrency pinned to the measured admission (2 of 6, 2026-08-24) [orlimits] [orrouting]; one ledger row per call (provider, model, prompt hash, bytes, class, terms pin, verdict) through nsp_hist_append; neg-control: a call whose terms row reads UNKNOWN is refused before the socket opens | Organ | 1.5 u |
| Public pack and external forge backend (XL4) after XL2,XL3 | fm_post_ext | fc_emit_public emits the rules corpus plus RELEASED stdlib signatures only and fails closed on any unreleased source exactly as fc_emit fails closed on an UNVERIFIED rule; fm_post_ext produces a candidate into the same file fg_run judges; gate: a candidate from the external tier passes the byte-exact judge on the L4 mini-pack task; neg-control: a pack carrying an unreleased signature is refused by name | Organ | 1.5 u |
| Best-of-N against the free verifier (XL5) after XL4 | fg_best_of_n | N candidates per task through fg_run, first GREEN kept, pass-at-N monotone in N on the mini-pack, NONE reported when no candidate is GREEN rather than the best-scoring wrong answer [llmonkeys]; the same contract /compare/coding names as CD2, so both pages flip on one organ | Organ | 1 u |
| External judge panel (XL6) after XL3 | lj_grade | free external models grade a U artifact under a pre-declared rubric and the grade travels with the claim as the independent referee the estate's progress law requires; the same contract the referee domain names [openaievals]; neg-control: a panel requested over a C artifact is refused at the door by XL2, never graded | Organ | 1.5 u |
| Rung worker loop (XL7) after XL4,XL5 | xl_rung_worker | reads a /compare plan rung whose spec and KAT resolve entirely to U sources, runs sampler then judge then bank, files the GREEN candidate on the pm intake plane for review by the assistant; a rung citing unreleased source is SKIPPED by name; the loop reports rungs attempted, GREEN, RED and SKIPPED with the partition summing | Organ | 2 u |
| Publish guard (XL8) after XL1 | xl_publish_guard | the compare emit may carry capability rows about an external provider and may NOT carry per-model performance numbers while the provider's terms row says benchmark_publish_restricted [stealtheula] [cerebrastos] [sambanovaeula]; measured numbers stay on the internal plane at class C; neg-control: an emit carrying a per-model score for a restricted provider is refused | Gate | 0.5 u |
| Terms-gated distillation harvest (XL9) after XL1,XL3 | xl_distill_corpus | SOVEREIGNTY IS THE ACCEPT RULE, NOT THE PERMISSION. The corpus must never DEPEND on a provider: with every external sampler disabled the verified ledger must still STRICTLY GROW over a run, and deleting every externally-provenanced row must leave it passing its quality bar unchanged. Both are measurable and a harvest that cannot show them is not wired. External output is then a SUPPLEMENT that can only ever enter as a CANDIDATE judged by our own byte-exact verifier, with its provenance recorded per row so the deletion test above stays possible. Only then does the terms gate apply: worker outputs are harvested into the forge's compile-verified ledger only where the terms row reads outputs_train_ours YES -- Groq [groqsa], Cloudflare Workers AI [cfterms], DeepSeek which permits distillation by name [deepseektos]; providers whose terms forbid training a competing model -- Gemini unpaid [geminiterms], NVIDIA trial [nvidiatrial], Z.ai [zaitos] -- are refused for this use and stay worker or judge only; UNKNOWN refuses | Organ | 1.5 u |
Milestones
| Milestone | Rungs | Cumulative |
|---|---|---|
| M0 · Classified call lane | XL1,XL2,XL3 | 4 u |
| M1 · Free worker on the forge | XL4,XL5 | 6.5 u |
| M2 · Independent referee and a safe page | XL6,XL8 | 8.5 u |
| M3 · Rung workers and lawful distillation | XL7,XL9 | 12 u |
| Class | Count | Note |
|---|---|---|
| bars | 0 | sotabar rows on the plan |
| fresh | 0 | the bar month is the current month |
| attested | 0 | an older bar with a this-month barscan reporting no newer entry |
| stale | 0 | a this-month scan saw a newer month the bar ignores |
| unattested | 0 | nobody looked this month |
| malformed | 0 | a sotabar row missing a field |
| duplicate | 0 | bar ids an earlier row already carries |
| bars, summed | 0 | RECONCILED: the parts sum to the total 0 |
| current month | 2026-10 | the month every bar is judged against |
Inspect a rung and its prerequisites
Declared nodes 9. Rank input binding: STALE_OR_INVALID_BINDING. Dependency order is authored. Implementation, acceptance evidence, authority and resource readiness are unverified. No action is recommended or dispatched here.
Plan SHA-256 9b330a86f98e0415009c96c9f9139eef4abb519e181866f11259981d9481ab3b. Target rows 0; role rows 0. Existing risks and release worklog retain their own scope; no node completion is inferred.
Runtime observation coverage: 0 of 9 rungs have current, identity-bound execution observations. Observation may report RED. Accepted completion remains unknown; source presence, qualification and deployment do not establish it.
Use Enter or Space on a rung to inspect its contract. Prerequisite links locate another rung in this list; open its summary to inspect it. Estimates are authored effort, not forecasts.
XL1 — Provider terms ledger
Prerequisites: None declared; this does not establish execution eligibility.
Contract:
xl_terms_ledgerAcceptance: one row per provider read from the PINNED mirror, never the live site: trains_on_input, outputs_train_ours, benchmark_publish_restricted, retention, card_required, rate caps; UNKNOWN is a value and REFUSES every gated use; neg-control: a provider with no mirror row must never read ALLOW [@ortos] [@orprivacy] [@orlimits]
Authored effort: 1. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.XL2 — Egress classifier and release manifest
Prerequisites: None declared; this does not establish execution eligibility.
Contract:
xl_egress_classAcceptance: every prompt byte resolves to a public-index cid or to a row of the operator-declared release manifest, else the call REFUSES naming the offending source; the manifest is data the operator writes, never a default; neg-control: a prompt assembled from an unreleased runtime .nx must never leave; positive control: a prompt built only from public compare pages must pass
Authored effort: 1.5. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.XL3 — Call, scheduler and egress ledger
Prerequisites: XL1 (acceptance unverified), XL2 (acceptance unverified)
Contract:
xl_callAcceptance: request built from a conf row (provider, model, effort, max tokens) and sent through nx_https_req_complete with the key read from the vault; the 429 attributed to its source (shared pool versus this key) and the body state named; one lane per free pool with backoff derived from the measured refusal rate and concurrency pinned to the measured admission (2 of 6, 2026-08-24) [@orlimits] [@orrouting]; one ledger row per call (provider, model, prompt hash, bytes, class, terms pin, verdict) through nsp_hist_append; neg-control: a call whose terms row reads UNKNOWN is refused before the socket opens
Authored effort: 1.5. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.XL4 — Public pack and external forge backend
Prerequisites: XL2 (acceptance unverified), XL3 (acceptance unverified)
Contract:
fm_post_extAcceptance: fc_emit_public emits the rules corpus plus RELEASED stdlib signatures only and fails closed on any unreleased source exactly as fc_emit fails closed on an UNVERIFIED rule; fm_post_ext produces a candidate into the same file fg_run judges; gate: a candidate from the external tier passes the byte-exact judge on the L4 mini-pack task; neg-control: a pack carrying an unreleased signature is refused by name
Authored effort: 1.5. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.XL5 — Best-of-N against the free verifier
Prerequisites: XL4 (acceptance unverified)
Contract:
fg_best_of_nAcceptance: N candidates per task through fg_run, first GREEN kept, pass-at-N monotone in N on the mini-pack, NONE reported when no candidate is GREEN rather than the best-scoring wrong answer [@llmonkeys]; the same contract /compare/coding names as CD2, so both pages flip on one organ
Authored effort: 1. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.XL6 — External judge panel
Prerequisites: XL3 (acceptance unverified)
Contract:
lj_gradeAcceptance: free external models grade a U artifact under a pre-declared rubric and the grade travels with the claim as the independent referee the estate's progress law requires; the same contract the referee domain names [@openaievals]; neg-control: a panel requested over a C artifact is refused at the door by XL2, never graded
Authored effort: 1.5. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.XL7 — Rung worker loop
Prerequisites: XL4 (acceptance unverified), XL5 (acceptance unverified)
Contract:
xl_rung_workerAcceptance: reads a /compare plan rung whose spec and KAT resolve entirely to U sources, runs sampler then judge then bank, files the GREEN candidate on the pm intake plane for review by the assistant; a rung citing unreleased source is SKIPPED by name; the loop reports rungs attempted, GREEN, RED and SKIPPED with the partition summing
Authored effort: 2. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.XL8 — Publish guard
Prerequisites: XL1 (acceptance unverified)
Contract:
xl_publish_guardAcceptance: the compare emit may carry capability rows about an external provider and may NOT carry per-model performance numbers while the provider's terms row says benchmark_publish_restricted [@stealtheula] [@cerebrastos] [@sambanovaeula]; measured numbers stay on the internal plane at class C; neg-control: an emit carrying a per-model score for a restricted provider is refused
Authored effort: 0.5. Executor kind: Gate. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.XL9 — Terms-gated distillation harvest
Prerequisites: XL1 (acceptance unverified), XL3 (acceptance unverified)
Contract:
xl_distill_corpusAcceptance: SOVEREIGNTY IS THE ACCEPT RULE, NOT THE PERMISSION. The corpus must never DEPEND on a provider: with every external sampler disabled the verified ledger must still STRICTLY GROW over a run, and deleting every externally-provenanced row must leave it passing its quality bar unchanged. Both are measurable and a harvest that cannot show them is not wired. External output is then a SUPPLEMENT that can only ever enter as a CANDIDATE judged by our own byte-exact verifier, with its provenance recorded per row so the deletion test above stays possible. Only then does the terms gate apply: worker outputs are harvested into the forge's compile-verified ledger only where the terms row reads outputs_train_ours YES -- Groq [@groqsa], Cloudflare Workers AI [@cfterms], DeepSeek which permits distillation by name [@deepseektos]; providers whose terms forbid training a competing model -- Gemini unpaid [@geminiterms], NVIDIA trial [@nvidiatrial], Z.ai [@zaitos] -- are refused for this use and stay worker or judge only; UNKNOWN refuses
Authored effort: 1.5. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.
Learning and practice paths
No structured learning path is declared for this plan. Existing research, roadmap and worklog remain available above.
Rung dossiers — every record that bears on a rung, in one place
nx_swcompare_evidence extllm writes it beside its stamp); every gate below reads UNRECORDED until it runs.knowledge/organ_gate.conf read from this working directory (1 rows) and from the estate root (196 rows); rows in both counted once: 0; declarations read: 197. Every declaration in either copy counts, so a stub beside the publisher can never hide the root.XL1 Provider terms ledger · cell LANDED · gate NO-GATE-DECLARED
| Contract | xl_terms_ledger · executor Organ · 1 u · stands on nothing |
|---|---|
| Definition of done | one row per provider read from the PINNED mirror, never the live site: trains_on_input, outputs_train_ours, benchmark_publish_restricted, retention, card_required, rate caps; UNKNOWN is a value and REFUSES every gated use; neg-control: a provider with no mirror row must never read ALLOW [ortos] [orprivacy] [orlimits] |
| Matrix cell | TERMS: Provider terms ledger as data (retains, trains, output rights, benchmark-publish restriction, rate caps) read from the pinned mirrors and consumed by the guard · runtime/nx_extllm.nx · LANDED decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_extllm; declare nx_extllm TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
XL2 Egress classifier and release manifest · cell LANDED · gate NO-GATE-DECLARED
| Contract | xl_egress_class · executor Organ · 1.5 u · stands on nothing |
|---|---|
| Definition of done | every prompt byte resolves to a public-index cid or to a row of the operator-declared release manifest, else the call REFUSES naming the offending source; the manifest is data the operator writes, never a default; neg-control: a prompt assembled from an unreleased runtime .nx must never leave; positive control: a prompt built only from public compare pages must pass |
| Matrix cell | CLASSIFY: Egress classifier at the door -- every prompt byte must resolve to a U source (a public-index cid or a public-release manifest row) or the call REFUSES by name · runtime/nx_extllm.nx · LANDED decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_extllm; declare nx_extllm TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | 2026-09-10 measure 2026-09-10 LIVE DELIVERY: native nx_extllm missing-ledger fix guarded-promoted via nx_organ_ship job1789073942, source16350B SHA8ba255bf46f585c1810b45ab736cbdc65958a0f6b7c20677e9764882a779ea53; live61520B SHA117a970f11b724a3fb2d853654facdcb1d8f2fe0112be97250313b046e09bebc. Pre-adoption actualCLI3/3 job3917 and content144 runs lost0 behaviorIDENTICAL3918 reviewed. Ship explicit gate3/3, matched liveSHA, refreshed offc twin, already registered; gate closure banking unproven direct-edge independence disclosed, URLverify skipped for CLI. Root then exercised registered live MCP/API:3960 missing journal exit3 evidence-unavailable with null counts;3961 actual empty exit0 zero;3964 populated fixture exit0 provider-filtered42bytes from140B ledger. This measures disclosed bytes, NOT provider token use. Generic short-read/malformed-ledger protection and native subscription dispatch remain incomplete. No provider call or paid API fallback occurred. 2026-09-10 measure 2026-09-10 PRIVATE HANDLER QUALIFICATION: existing ClaudeSubscriptionTasks owner now exposes typed status/start/result method handlers with host-injected authorization and included-Max-only policy. Root independently reran 24/24 checks including actual retained result and missing evidence, with provider calls disabled. Root caught valid directory without trailing separator failing; existing facade now normalizes it once, both forms tested. Source32133B SHA098a7c682dbf87bf544d15da2d7d9c23ea8f1e12b0d2d3b550469b9619ffb899. Evidence knowledge/gates/extllm-directory-normalization-20260910.json and extllm-typed-tools-qualification-20260910.json. Dedicated authenticated host registration remains OPEN; synthetic handler mounting is not native registration. No new service or paid provider fallback. Next integrate with existing host and retain actual provider dispatch receipts. 2026-09-10 measure 2026-09-10 ACTUAL SUBSCRIPTION DISPATCH: one public synthetic rational-clipping task ran on verified personal-aster Max, Opus5 selected after fresh Fable100percent-used refusal; extra usage disabled. Root read full prompt, provider result and review: exact crossing formula and three fixture answers accepted; collinear-degenerate handling and overflow-sizing guidance remain incomplete, not admitted as implementation. ModelUsage categories including ancillary Haiku:1279 input,3809 output,3284 cache-created,0 cache-read;8372 category sum is not subscription quota units. Provider estimate7000 under by1372 (19.6percent); actual root review tokens and net savings unknown. Fresh before/after weekly-used55percent unchanged at display precision, not proof zero consumption. List-price estimate is not billed spend. Evidence knowledge/gates/extllm-rational-clip-review-20260910.json;1 provider attempt,0 retries,0 deployments. Typed method call through Node MCP is verified; dedicated native host registration remains OPEN. 2026-09-10 measure 2026-09-10 TOKEN ROI INVESTMENT: operator prioritizes quality and timeliness, using included personal Claude Max capacity without paid API/overage fallback. Net root-token saving remains UNKNOWN: observed provider tokens are available for one latest reviewed task, but actual briefing/review/rework token attribution and a matched root-only counterfactual are absent. Do not turn unknown into zero or claim positive savings from completed responses. Existing delegation owner must retain task class, account/model identity, measured provider categories, actual supervision where observable, accepted/reworked/rejected outcome, integration evidence and elapsed time; counterfactual avoided effort must be labeled estimated with its comparison method. Rank reusable task classes by accepted integrated work with quality and delivery-time gates. Subscription quota percentages are separate from token counts; ancillary model usage remains attributed. No new provider dispatch or public Compare regeneration is established by this record. 2026-09-10 measure 2026-09-10 CONTEXT-INCLUSIVE ROOT USAGE: local current-root rollout token_usage_record exposes request response_id/thread_id/turn_id and usage categories. Independently deduplicated root window [21:48:00,21:52:00)UTC contains24 requests: input4279186, cached-input4229632 (subset), output4508, reasoning1216 (output subset), total4283694; noncached-input49554, cache-hit98.841976percent. All24 total=input+output and subset checks pass; zero duplicate responseIDs in window. This is mixed Beach/delegation/estate work, NOT dedicated Claude supervision; other Codex workers not included. Latest Claude calibration5167 reportedcategorysum accepted8/8 with no revision. No billed-cost, quota-conversion or net-savings claim. Retained request-level evidence C:/Users/elder/nishi-ops/root-token-window-20260910-2148-2152.json. Next existing usage owner ingests request records with explicit window/coverage and task-attribution evidence, keeping shared context cost visible and unknown attribution null. 2026-09-10 measure 2026-09-10 LOG STORAGE AND ARCHIVE AUDIT: metadata-only local scan measured Codex sessions1668829443B, nishi-ops1642700136B, .nishi9751126B; C available49379598336B of994834378752B. This is logical size in three directories, not full disk/NAS usage or causal attribution of disk pressure. Existing nx_jrnl_archive source has uncoordinated copy-then-truncate and overwrite-capable timestamp archive names; fsync results ignored. Existing nx_dr_archive caps source reads64MiB and can verify only that prefix. Existing retention policy is report-only and does not reclaim disk. Full source findings and dated official research retained in knowledge/gates/archive-source-audit-20260910-t132.json. No production archive, truncation or deletion executed. Repair existing owners; streaming immutable segments, checked durability, full restore verification and query continuity required before archival qualification. Compression alone does not bound retention; do not replicate full transcripts into accounting receipts. 2026-09-10 invest 2026-09-10 ROI direction: optimize accepted integrated capability per total root+worker investment, preserving or improving quality and delivery speed; spending more is acceptable for superior returns. Existing usage owner must separate request task/phase (implementation,research,review,retry,coordination,mixed-unknown) from input composition (new/repeated context,tool results,cached input) and output; overlapping categories never sum as disjoint spend. Rank native investment:1 artifact-backed compact tool results and reviewed-delta retrieval;2 scoped delegation packets with durable task state and recoverable workflows;3 measured prompt/retrieval/model-routing optimization. Full evidence remains retrievable; omitted failures, stale hashes, missing context and unknown coverage must fail acceptance. Research checked2026-09-10: RTK command-output reduction https://github.com/rtk-ai/rtk ; Pi runtime/state/telemetry https://github.com/earendil-works/pi ; DSPy GEPA metric-guided instruction optimization https://github.com/stanfordnlp/dspy/blob/main/docs/docs/diving-deeper/gepa-in-depth.md ; Microsoft LLMLingua https://github.com/microsoft/LLMLingua plus independent2026 evaluation https://arxiv.org/abs/2604.02985 . These are candidates, not measured Nishi savings or adopted dependencies. Benchmark matched Beach/swarm tasks on acceptance,missed defects,rework,wall time,root+worker tokens and resource use; retain setup cost and uncertain counterfactual. Lossy compression needs stronger exact-detail tests. Subscription-only/no paid-overage unchanged. Nishi search exact query context engineering returned unrelated engine results at top; coverage incomplete, so external primary-source research used. Reuse current catalogue/collector/dispatcher owners; no duplicate service or automatic training authorization. Public Compare UI regeneration not verified by this log. 2026-09-10 invest 2026-09-10 EVIDENCE-PRESERVING CAPABILITY ROI: operator requires lower token burden while intelligence,effectiveness and capability grow; details that reveal subtle defects must remain discoverable. Extend existing XL2 investment domain rather than new overlapping board. Compare raw-evidence baseline against proposed compact/delta workflow using retained cases: garment kind8 family4/5/8/9 selection divergence; mismatched shader source/layout; native usage conflicting duplicates and unknown evidence; whole-event leakage through local accounting normalization; persistent-state corruption. Acceptance requires catching the relevant defect with source-backed explanation, retrieving exact omitted bytes by immutable identity, and preserving missing/truncated/error markers. Measure root+worker tokens,coordination,retries,wall time,accepted integration,missed defects and one-time capability build cost separately; count cross-project reuse without double-counting outcome value. No compression percentage or lower model cost overrides quality/time requirements. Summarization is a navigation aid, never proof that unread source was reviewed. Current research/implementation remains partial; no optimality or public board-render claim. 2026-09-10 measure 2026-09-10 request accounting delivered through existing nx_claude_harvest usage-window <source.jsonl> <start-inclusive-epoch> <end-exclusive-epoch> <scan-start-byte> <scan-end-byte> and canonical local extllm_account_admission CodexRequestAccounting.observe. Native ship8654/call8688 verified44 requests7928230 tokens; direct original-log root24/4283694 and dispatcher20/3644536 match retained windows, canonical replay emits zero new requests/native calls. Adapter50202B SHAe8c68f57ac706d1b15f514df0348a451e5ea25a61df9872021a128820f365a51; existing checks owner preserves15 admission cases and reusable12 boundary cases pass. Whitelisted counters only, authenticated bounded cursor state, mixed evidence-linked phase annotations; billing/quota/task token partition/savings unknown. Receipt C:/Users/elder/nishi-ops/request-accounting-t76/canonical-local-delivery.json. Scheduler/UI, causal attribution, archive lifecycle and power-loss directory durability remain open. Research/optimization requirements reuse XL2 logs1789078983 and1789079136. 2026-09-10 measure Cross-estate capability ROI acceptance: compare matched-task baseline against all root, worker, review, retry and context effort; cached/new input/output remain orthogonal. Require quality, defect discovery, recovery and time-to-accepted-result gates. Exact raw evidence remains addressable with explicit omissions and uncertainty; missing usage is unknown. Conditional identity reader is first bounded intervention, not proof of preserved reasoning quality or ROI from smaller payload alone. Retained qualification compact-evidence-t77; private reader18/18 native, broader matched-task acceptance pending. 2026-09-10 measure Timestamp correction for preceding ROI acceptance row1789081960: its timestamp was clerical; actual append job1789081912 completed and was read back by observed2026-09-10T23:12:03Z. Acceptance content unchanged; prior record preserved. 2026-09-10 measure Confirmed native syscall correctness gap found during evidence-preserving reader qualification: nx_x86_64_ctx.nx x86ctx_rv64_to_x86_64_syscall line2252 maps RV64 pipe2(59) to x86 pipe(22), dropping NONBLOCK/CLOEXEC flags. Actual emitted buildroot/_build/nx_fsops_conditional_gate_t77.s sys_pipe2 sets rax22; job1789082125 timed out on intended nonblocking pipe write. Exact excerpt retained compact-evidence-t77/pipe2-emitted-mapping-refusal.txt. No compiler mutation; follow-up needs owning translation correction and real flags0/NONBLOCK/CLOEXEC, invalidflags, both-end F_GETFL/F_GETFD, and nonblocking-empty-read regression before compiler adoption. Current reader gate uses existing nx_fcntl_set_nonblock plus checked F_GETFL; this does not close pipe2 defect. |
XL3 Call, scheduler and egress ledger · cell LANDED · gate NO-GATE-DECLARED
| Contract | xl_call · executor Organ · 1.5 u · stands on XL1, XL2 |
|---|---|
| Definition of done | request built from a conf row (provider, model, effort, max tokens) and sent through nx_https_req_complete with the key read from the vault; the 429 attributed to its source (shared pool versus this key) and the body state named; one lane per free pool with backoff derived from the measured refusal rate and concurrency pinned to the measured admission (2 of 6, 2026-08-24) [orlimits] [orrouting]; one ledger row per call (provider, model, prompt hash, bytes, class, terms pin, verdict) through nsp_hist_append; neg-control: a call whose terms row reads UNKNOWN is refused before the socket opens |
| Matrix cell | CALL: Chat-completions call to an external provider over the sovereign TLS client · runtime/nx_extllm_call.nx · LANDED decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_extllm_call; declare nx_extllm_call TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | 2026-09-10 land 2026-09-10 PARTIAL MEASUREMENT CAPABILITY: existing nx_claude_harvest source adopted with predecessor retained; native60/60 regression checks passed and serving binary270544B SHA25610bf79c136ad63cfa1290279e1019541641bc5f9bb9baa9d602a51cc246e07d0 deployed by existing build lane. Direct API jobs1789052862/2863 distinguish missing-source exit3 from realempty exit0. Additive evidence_state and measurement_scope label observed tool frames; actor work share, token cost and verified delivery share remain null. Pending: shared read-integrity guarantees, response/message-ID token accounting, task-to-artifact acceptance joins, subscription dispatch and visible Compare regeneration. This does not complete XL3 or prove SOTA. 2026-09-10 land 2026-09-10 PARTIAL TEAM FOUNDATION: reused existing nx_estate_path.ep_anchor in actual nx_orchestrate_raci CLI and existing gate; no duplicate store or resolver. Read-only context probe4290 passed6/6: buildroot missing owners, serving root librarian/researcher resolved, unknown refused. Canonical gate4418 passed13/13 on live RACI and engagement store; emitted label corrected to resolve+plan+record, explicitly no workstream dispatch. Gate auto-refreshed146872B SHA3035a4b74da56b0a71c47da7c2d47e834d8aeca1d8365b1b6c5f2f574f7799ac. Actual CLI4419 build125896B SHAae42b09cf2f658d8d06bd81f6256c3e800ad04ac7d08dc86333a512a6d5f2862 planned organize successfully; no standalone deploy line, so standalone publication unverified. Missing bindings, evidence-write error propagation, account controls, advanced team lifecycle, portable configurable root and API registration remain open. Browser/video existing requirements remain scope; Beach agents continue private body and workspace integration. 2026-09-10 land 2026-09-10 PARTIAL TEAM OUTCOME INTEGRITY: actual nx_orchestrate_raci research run previously exited0 PLANNED with no workstream bound (4635). Corrected existing owner10367B, backup retained, adopted4735. Canonical build/run4744 now exits1 NOT-RUN-UNBOUND with reason/next action;126240B SHA96ee6f50791b771fd43dd5c7e94164a233978685ebfa53f1b6bbd9724592ea17. Private evidence-write failure control4708 used impossible directory under /dev/null, observed store lock open failure and exit1 evidence=write-failed without saved-record claim. Normal organize plan4715 remains exit0. Calls used sovereign build/API, no standalone dispatcher deployment line observed. These are failure-reporting and planning capabilities, not delivered autonomous research/advanced team execution; qualified workflow binding and native provider dispatch remain open. Beach page and body wiring continue in parallel. 2026-09-13 watch 2026-09-13 DeepSeek trial preparation: native nx_extllm terms call verified job1789320866; nx_extllm_call registered (historical NO external call organ text is stale). Requested 4.1 identity, no-paid route and account data controls unverified; no inference, no harness install, no proprietary disclosure. Public synthetic cohort and total-cost unknowns retained in buildroot/knowledge/gates/deepseek-trial-preparation-t273.json. Hidden executable gate and blinded evaluation remain prerequisites, not delivered results. |
XL4 Public pack and external forge backend · cell WATCHING · gate NO-GATE-DECLARED
| Contract | fm_post_ext · executor Organ · 1.5 u · stands on XL2, XL3 |
|---|---|
| Definition of done | fc_emit_public emits the rules corpus plus RELEASED stdlib signatures only and fails closed on any unreleased source exactly as fc_emit fails closed on an UNVERIFIED rule; fm_post_ext produces a candidate into the same file fg_run judges; gate: a candidate from the external tier passes the byte-exact judge on the L4 mini-pack task; neg-control: a pack carrying an unreleased signature is refused by name |
| Matrix cell | WORKER: External model backend for the forge with the same candidate contract as the local serve · runtime/nx_forge_model.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_forge_model; declare nx_forge_model TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
XL5 Best-of-N against the free verifier · cell WATCHING · gate NO-GATE-DECLARED
| Contract | fg_best_of_n · executor Organ · 1 u · stands on XL4 |
|---|---|
| Definition of done | N candidates per task through fg_run, first GREEN kept, pass-at-N monotone in N on the mini-pack, NONE reported when no candidate is GREEN rather than the best-scoring wrong answer [llmonkeys]; the same contract /compare/coding names as CD2, so both pages flip on one organ |
| Matrix cell | WORKER: Best-of-N sampling against the free verifier with the external tier as a sampler · runtime/nx_forge.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_forge; declare nx_forge TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | 2026-09-13 land 2026-09-13 Canonical nx_forge_judge now has eval-release for fixture and response JSON argument values (not file paths). Native helper gate16/16; canonical CLI1460 passed escaped equivalent key and preserves unknown build evidence; usage1480 returns expected2. Structured fixture/response SHA256 receipts, no substring acceptance for this command, never release authority. Callable through existing mgmt native qualification; named route registration remains unqualified. No provider calls or ROI claim. Evidence buildroot/knowledge/gates/forge-evidence-adoption-t274.json. |
XL6 External judge panel · cell WATCHING · gate NO-GATE-DECLARED
| Contract | lj_grade · executor Organ · 1.5 u · stands on XL3 |
|---|---|
| Definition of done | free external models grade a U artifact under a pre-declared rubric and the grade travels with the claim as the independent referee the estate's progress law requires; the same contract the referee domain names [openaievals]; neg-control: a panel requested over a C artifact is refused at the door by XL2, never graded |
| Matrix cell | JUDGE: External LLM-as-judge panel over U content, the independent referee grade recorded beside every claim it grades · runtime/nx_llm_judge.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_llm_judge; declare nx_llm_judge TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
XL7 Rung worker loop · cell WATCHING · gate NO-GATE-DECLARED
| Contract | xl_rung_worker · executor Organ · 2 u · stands on XL4, XL5 |
|---|---|
| Definition of done | reads a /compare plan rung whose spec and KAT resolve entirely to U sources, runs sampler then judge then bank, files the GREEN candidate on the pm intake plane for review by the assistant; a rung citing unreleased source is SKIPPED by name; the loop reports rungs attempted, GREEN, RED and SKIPPED with the partition summing |
| Matrix cell | WORKER: Rung worker loop -- reads a /compare plan rung whose spec and KAT are public, runs sampler then judge then bank, files the candidate for review · runtime/nx_extllm.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_extllm; declare nx_extllm TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | 2026-09-10 research 2026-09-10 operator supplied post-training/inference research added to candidate map, not implementation authorization. Primary checks: DeepSeekMath https://arxiv.org/abs/2402.03300 defines GRPO as PPO variant with group-relative rewards and reduced critic-memory requirements; it does not inherently optimize cleanest code. Phi4 https://arxiv.org/abs/2412.08905 credits data quality, curriculum and post-training with minimal architecture changes; task specialization need not remove general abilities. RLVR rewards only what its verifier covers; passing tests is not absence of defects. Immediate Beach worker investment stays inference-only: bounded task, context/tool contract, independent executable oracle, retained actual artifact and interaction evidence, resource/error measurements and calibrated escalation. First Fable public arithmetic fixture passed3 independent assertions, a calibration only; no general coding or deployment authority. No Nishi training dataset, fine-tuning or distillation initiated. New worker research refs in existing extllm.refs remain accessed-but-unpinned, not SOTA qualification. Continue comparative experiments across held-out real task families before raising worker authority. 2026-09-10 measure 2026-09-10 bounded supervised Claude usage baseline:2 completed Fable5.1 public calibration tasks on1 of3 authorized accounts (33.33% connections verified); work account excluded. Provider message IDs deduplicate3 assistant transcript rows to2 calls. Sum categories exactly once per message: uncached input4, cache creation13580, cache read66344, output863 including provider-reported thinking subset355; total token events80791. Shares: cache read82.12%, creation16.81%, output1.07%, uncached input0.005%. These are provider token events, not billable-dollar conversion or newly generated tokens. Both numeric fixture results independently checked;2/2 calibration outcomes is NOT general coding reliability. Zero Claude production deployments. Latest observed account allowance52%weekly and95%Fable used, usage credits off; percentages are account-specific provider observations, not estate task share. Nishi research3 attempted fetch invocations:2 actual source fetches succeeded with retained provenance,1 unsupported --help input failed; all must remain in accounting. Goal counter15372722 at its recorded observation is cumulative thread accounting, not a proven root-only/provider-normalized total. Cross-provider token/work percentages remain UNKNOWN until common time window, message deduplication, parent-child task attribution and executed/verified/delivered outcomes are joined. Existing Compare/model-worker and transcript-harvest owners must consume the evidence, not a new competing ledger. |
XL8 Publish guard · cell LANDED · gate NO-GATE-DECLARED
| Contract | xl_publish_guard · executor Gate · 0.5 u · stands on XL1 |
|---|---|
| Definition of done | the compare emit may carry capability rows about an external provider and may NOT carry per-model performance numbers while the provider's terms row says benchmark_publish_restricted [stealtheula] [cerebrastos] [sambanovaeula]; measured numbers stay on the internal plane at class C; neg-control: an emit carrying a per-model score for a restricted provider is refused |
| Matrix cell | PUBLISH: Publication guard -- external-model performance numbers stay C, capability rows stay U · runtime/nx_extllm_publish_guard.nx · LANDED decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_extllm_publish_guard; declare nx_extllm_publish_guard TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
XL9 Terms-gated distillation harvest · cell WATCHING · gate NO-GATE-DECLARED
| Contract | xl_distill_corpus · executor Organ · 1.5 u · stands on XL1, XL3 |
|---|---|
| Definition of done | SOVEREIGNTY IS THE ACCEPT RULE, NOT THE PERMISSION. The corpus must never DEPEND on a provider: with every external sampler disabled the verified ledger must still STRICTLY GROW over a run, and deleting every externally-provenanced row must leave it passing its quality bar unchanged. Both are measurable and a harvest that cannot show them is not wired. External output is then a SUPPLEMENT that can only ever enter as a CANDIDATE judged by our own byte-exact verifier, with its provenance recorded per row so the deletion test above stays possible. Only then does the terms gate apply: worker outputs are harvested into the forge's compile-verified ledger only where the terms row reads outputs_train_ours YES -- Groq [groqsa], Cloudflare Workers AI [cfterms], DeepSeek which permits distillation by name [deepseektos]; providers whose terms forbid training a competing model -- Gemini unpaid [geminiterms], NVIDIA trial [nvidiatrial], Z.ai [zaitos] -- are refused for this use and stay worker or judge only; UNKNOWN refuses |
| Matrix cell | DISTILL: Synthetic corpus harvest for the no-float LM, gated per provider on the output-rights row of the terms ledger · runtime/nx_extllm.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_extllm; declare nx_extllm TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
dossiers=9 cells_joined=9 no_matrix_row=0 (sum 9) gates_recorded=0 gates_unrecorded=0 no_gate_declared=9 (sum 9 of cells_joined) evgates=UNSTAMPED
Capability comparisons
Compare the field, search individual capabilities and open their source and adoption evidence. Documented presence does not establish comparative quality.
Position map — centrality and distinctiveness
The four-quadrant map the field uses for brand strategy (Dawar and Bagga, HBR June 2015), re-derived from this matrix on every publish. Centrality is the share of the category's feature mass a player covers, each feature weighted by how many hold it; distinctiveness is the average lead over each rival on the rows the player holds; breadth is the depth-weighted share of the whole matrix (the bubble); depth is how deeply the rows held are held; momentum is the day-over-day move off the spine (green rising, red falling, grey until day two); the dashed path runs first day → previous day → today. Dividers are the category means. Axes are fitted to the field of play, so read the tick numerals, not the frame. Rival marks are documented presence, so a rival's position reads the record, never its quality. The picture grades its own readability below; the table beside it is the same data for a screen reader or a second method.
knowledge/compare/extllm.cdmap (cut|x|y, cube|x|y|z); absent = the HBR defaultsbubble area = breadth · ring = momentum (green rising, red falling, grey until day two) · dashed = category means · axes fitted to the field of play: centrality 0–1000, distinctiveness 100–800 of 0–1000 permil (the full range put every player in one corner)
readability of the 2D cut, self-graded by the layout ruler: label overlaps 0 · labels over marks 0 · off-canvas 0 · unresolved labels 0 · mark overlaps 0 (a fact of the data: two players that close are that close) · data spread 646 permil of the plot · quadrant words unseated 0
text contrast, measured with wcag2-ratio (floors from contrast.conf), light theme: labels 16.24 (floor 4.50) · notes 16.24 (floor 4.50) · quadrant words 5.89 (floor 4.50) · tick numerals 5.89 (floor 4.50) · axis titles 5.59 (floor 4.50) · dark theme: labels 13.78 (floor 4.50) · quadrant words 5.55 (floor 4.50) · axis titles 5.97 (floor 4.50) · dark classes under their floor 0 (one figure serves both themes: a dark shortfall is a token to fix, never a class to hide) · classes refused under their floor 0 (a refused class is not drawn; the scale classes are measured, never hidden) · export: SVG PNG (receipt, rendered by the estate's own rasteriser from this page)
readability of the 3D cube, self-graded by the layout ruler: label overlaps 0 · labels over marks 0 · off-canvas 1 · unresolved labels 0 · mark overlaps 0 (a fact of the data: two players that close are that close) · data spread 137 permil of the plot
10 panels over 5 registered axes, every pair once (the lower axis on x, the higher on y) · each panel fitted to its own field of play, first and last tick numerals shown · no labels in a small cut, the legend names the colours; the grade below reports the mark terms only (MM summed over the panels, spread averaged)
readability of the small multiples, self-graded by the layout ruler: label overlaps 0 · labels over marks 0 · off-canvas 0 · unresolved labels 0 · mark overlaps 2 (a fact of the data: two players that close are that close) · data spread 542 permil of the plot
| Player | Quadrant | centrality | distinctiveness | breadth | depth | momentum | Rows held | Days on spine | First seen |
|---|---|---|---|---|---|---|---|---|---|
| Nishi | Aspirational | 826 | 655 | 559 | 787 | 0 since 2026-09-23 | 22 | 10 | 2026-09-15 |
| LiteLLM gateway | Mainstream | 576 | 363 | 225 | 636 | 0 since 2026-09-23 | 11 | 10 | 2026-09-15 |
| OpenRouter router | Peripheral | 442 | 333 | 161 | 625 | 0 since 2026-09-23 | 8 | 10 | 2026-09-15 |
| LLM Guard (egress DLP) | Peripheral | 153 | 222 | 43 | 444 | 0 since 2026-09-23 | 3 | 10 | 2026-09-15 |
| openai-evals (model grading) | Unconventional | 346 | 468 | 172 | 666 | 0 since 2026-09-23 | 8 | 10 | 2026-09-15 |
| Day | Matrix rows reviewed | Players recorded |
|---|---|---|
| 2026-09-15 | 31 | 5 |
| 2026-09-16 | 31 | 5 |
| 2026-09-17 | 31 | 5 |
| 2026-09-18 | 31 | 5 |
| 2026-09-19 | 31 | 5 |
| 2026-09-20 | 31 | 5 |
| 2026-09-21 | 31 | 5 |
| 2026-09-22 | 31 | 5 |
| 2026-09-23 | 31 | 5 |
| 2026-10-08 | 31 | 5 |
players 5|matrix rows 31|feature mass 52|centrality mean 468|distinctiveness mean 408|axes 5|spine days 10 (shown 10)|rows written today 0|readability defects 2D 0 cube 1|spine knowledge/status/cdmap/extllm.spine
Capability matrix — measured against source
◉ authored stronger claim; source present● present◐ partial○ absent? unassessed; excluded from relative comparison · click any capability for its evidence
Peer cell coverage: authored assessed 124, unassessed 0. Authored assessments are not verified outcomes.
No capabilities match this filter or search.
| Capability | Nishi | LiteLLM gateway | OpenRouter router | LLM Guard (egress DLP) | openai-evals (model grading) |
|---|---|---|---|---|---|
| TRANSPORT | |||||
HTTPS JSON POST over the sovereign TLS 1.3 client (own stack, Mozilla trust store)Source present; authored stronger claim:yt_post in runtime/_hdl_build/nx_yt_innertube.nx, verified at emit. The gateway and the guard ride a Python HTTP client over a system TLS library [litellm] [llmguard]; ours is the estate's own TLS 1.3 client, proven on the InnerTube POST path -- no third-party TLS sits between the estate and the provider Adoption: LIB-WIRED importers=1 nonval=1 — fully adopted (top of its ladder).Pros Source symbol found; stronger capability claim is authored; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ◉ | ● | ● | ● | ○ |
Prebuilt request sent over an established TLS session (the primitive a chat-completions call composes)Measured:nx_https_req_complete exists in runtime/nx_https_get_complete.nx, verified at emit. Fourteen call sites in the tree already compose it; the external call is a request builder plus this one function, not a new transport rival marks uncited — the Yes, Best or Part codes on this row are an observation read with no reference mark behind them Adoption: SOURCE-ONLY — PARTIAL: source exists, never compiled: /api/build it.Pros Implementing source symbol found; runtime acceptance separate; ahead of LLM Guard (egress DLP), openai-evals (model grading)Cons not adopted yet: source exists, never compiled: /api/build it | ● | ● | ● | ○ | ○ |
Body-completeness state named on every response (COMPLETE, TRUNCATED, UNJUDGEABLE, UNOBSERVED)Source present; authored stronger claim:nx_https_body_state in runtime/nx_https_get_complete.nx, verified at emit. A reply cut at the wire reads as a short answer in every client; ours names the state. Measured on the subject 2026-08-24: the shared free pool answered 429 on roughly half of first attempts and admitted 2 of 6 concurrent calls [oxalpha] Adoption: SOURCE-ONLY — PARTIAL: source exists, never compiled: /api/build it.Pros Source symbol found; stronger capability claim is authored; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading)Cons not adopted yet: source exists, never compiled: /api/build it | ◉ | ○ | ○ | ○ | ○ |
| SECRETS | |||||
Provider key sealed at rest (AES-128-GCM, KDF-stretched passphrase)Measured:vlt_derive exists in runtime/_hdl_build/nx_vault.nx, verified at emit. The gateway keeps provider keys in environment and config [litellm]; ours is the vault -- a key never touches argv, source, a memory file or a page Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked); only referenced by nx_vault_v3_gate -- a mention, not a run.Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading)Cons not adopted yet: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked); only referenced by nx_vault_v3_gate -- a mention, not a run | ● | ◐ | ○ | ○ | ○ |
Lease-driven key rotation with a grace overlapMeasured:rot_due exists in runtime/_hdl_build/nx_vault_rotation.nx, verified at emit. A key pasted into a chat is exposed the moment it is pasted; rotation here is a lease that falls due, not a reminder rival marks uncited — the Yes, Best or Part codes on this row are an observation read with no reference mark behind them Adoption: LIB-GATE-ONLY importers=2 — PARTIAL: imported only by validation organs (gates, tests, benches): wire it into a shipping program.Pros Implementing source symbol found; runtime acceptance separate; ahead of OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading)Cons not adopted yet: imported only by validation organs (gates, tests, benches): wire it into a shipping program | ● | ● | ○ | ○ | ○ |
| CLASSIFY | |||||
Reader clearance resolved from the session uid (U, C, S, TS)Source present; authored stronger claim:ag_uid_to_level in runtime/_hdl_build/nx_access_lib.nx, verified at emit. The estate's Bell-LaPadula model gives every reader a level; an external provider is a sink with clearance U. The egress DLP field scans for secret and PII patterns [llmguard] and has no notion of a clearance -- a pattern list is a floor, a level is a rule Adoption: LIB-WIRED importers=16 nonval=15 — fully adopted (top of its ladder).Pros Source symbol found; stronger capability claim is authored; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ◉ | ○ | ○ | ◐ | ○ |
The public U index served with a per-document consent bit (the only material a U sink may receive)Source present; authored stronger claim:dss_search_off_div in runtime/_hdl_build/nx_docportal_search_seg.nx, verified at emit. No-write-down by construction: what the anonymous public search already serves is, by definition, already public. Anything else is C by default and never leaves Adoption: LIB-WIRED importers=22 nonval=13 — fully adopted (top of its ladder).Pros Source symbol found; stronger capability claim is authored; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ◉ | ○ | ○ | ○ | ○ |
| EVIDENCE | |||||
Provider terms mirrored with provenance (url, sha256, bytes) on every fetchMeasured:rf_provenance exists in runtime/nx_research_fetch.nx, verified at emit. The Stealth EULA [stealtheula], the OpenRouter terms [ortos] and the model page [oxalpha] are mirrored and pinned, not remembered Adoption: RUN-BY:actlog — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ○ | ○ | ○ | ○ |
Citations register gate, pin must equal the live hashMeasured:crg_pin_ok exists in runtime/nx_compare_refs_gate.nx, verified at emit. A terms page that changes under you re-pins RED on the next emit; the terms ledger contract below reads the pinned mirror, never the live site Adoption: GATE:LIVE trial=- — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ○ | ○ | ○ | ○ |
| JUDGE | |||||
Byte-exact deterministic build plus judge of a candidateSource present; authored stronger claim:fg_run in runtime/nx_forge.nx, verified at emit. The worker's output is judged by an equality, never by a model grading a model; model-graded evals [openaievals] are the field's shape and the referee domain records that gap honestly Adoption: LIB-WIRED importers=8 nonval=5 — fully adopted (top of its ladder).Pros Source symbol found; stronger capability claim is authored; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ◉ | ○ | ○ | ○ | ◐ |
Bounded repair loop feeding verbatim compiler stderr back to the workerMeasured:fg_bank exists in runtime/nx_forge.nx, verified at emit. The loop the local Coder tier already runs; the external tier plugs into the same bank Adoption: LIB-WIRED importers=8 nonval=5 — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ○ | ○ | ○ | ○ |
| WORKER | |||||
Prompt composed from the context pack plus the task plus repair feedbackMeasured:fm_compose_prompt exists in runtime/nx_forge_model.nx, verified at emit. The prompt is assembled from named files, which is what makes the egress classifier below decidable: every byte has a source path rival marks uncited — the Yes, Best or Part codes on this row are an observation read with no reference mark behind them Adoption: LIB-WIRED importers=3 nonval=2 — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of OpenRouter router, LLM Guard (egress DLP); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ● | ○ | ○ | ● |
Fence stripping and text extraction on a model replyMeasured:fm_strip_fences exists in runtime/nx_forge_model.nx, verified at emit. Measured on the subject 2026-08-24: at max effort ox-alpha wrapped its code in fences despite an explicit no-fences instruction; this organ exists for exactly that rival marks uncited — the Yes, Best or Part codes on this row are an observation read with no reference mark behind them Adoption: LIB-WIRED importers=3 nonval=2 — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ● | ○ | ○ | ○ |
Context pack emitted from sources (rules corpus plus stdlib signatures plus exemplar)Measured:fc_emit exists in runtime/nx_forge_ctx.nx, verified at emit. Today's pack extracts signatures from estate source, which is C by default under classify-at-the-door; the public-only pack is a contract below and an operator decision Adoption: LIB-WIRED importers=7 nonval=4 — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ○ | ○ | ○ | ○ |
| LEDGER | |||||
Provenanced additive plane write with a history row per changeMeasured:nsp_hist_append exists in runtime/_hdl_build/nx_store_put.nx, verified at emit. The gateway logs spend per key [litellm]; the router shows activity per key [oxalpha]; ours writes every egress as a plane row with actor, hash and class -- the ledger contract below is one consumer of this verb Adoption: RUN-BY:clock — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ◐ | ◐ | ○ | ○ |
| BENCH | |||||
SWE-bench Verified grader owning the denominator (python is the oracle, never the scorer)Measured:sg_verdicts exists in runtime/_hdl_build/nx_swebv_grade.nx, verified at emit. The repository-level bar [swebench] is graded by an estate organ; an external worker's patches are graded by the same organ, so a free model's SWE-bench standing is a number we compute, not one we read Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked).Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP)Cons not adopted yet: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked) | ● | ○ | ○ | ○ | ● |
| GATE | |||||
Verdict base class, the exit code carries the verdictMeasured:gv_verdict exists in runtime/nx_gate_verdict.nx, verified at emit. Every contract below ships with a gate on this class, including a neg-control that a C-classified prompt is REFUSED rival marks uncited — the Yes, Best or Part codes on this row are an observation read with no reference mark behind them Adoption: LIB-WIRED importers=2826 nonval=854 — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ○ | ○ | ○ | ● |
| EVIDENCE | |||||
Sovereign mirrored and pinned reference register for this fieldMeasured:knowledge/fetched/cmp_extllm exists in knowledge/compare/extllm.refs, verified at emit. Every rival claim on this page resolves to a mirror in the estate evidence store, content-pinned by hash and re-measured on every emit Adoption: ABSENT — PARTIAL: no source and no binary: this is a build contract, not a partial deploy.Pros Implementing source symbol found; runtime acceptance separate; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading)Cons not adopted yet: no source and no binary: this is a build contract, not a partial deploy | ● | ○ | ○ | ○ | ○ |
| CALL | |||||
Chat-completions call to an external provider over the sovereign TLS clientMeasured:xl_call exists in runtime/nx_extllm_call.nx, verified at emit. The gateway [litellm] and the router [oxalpha] are the field; contract = request built from a conf row (provider, model, effort, max tokens), sent through nx_https_req_complete, the 429 attributed by source (upstream shared pool versus this key), the body state named Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked).Pros Implementing source symbol found; runtime acceptance separate; ahead of LLM Guard (egress DLP)Cons behind LiteLLM gateway (leads), OpenRouter router (leads); not adopted yet: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked) | ● | ◉ | ◉ | ○ | ● |
| CLASSIFY | |||||
Egress classifier at the door -- every prompt byte must resolve to a U source (a public-index cid or a public-release manifest row) or the call REFUSES by nameSource present; authored stronger claim:xl_egress_class in runtime/nx_extllm.nx, verified at emit. The DLP field redacts patterns it recognises [llmguard]; ours refuses anything it cannot prove public. Neg-control: a prompt assembled from a runtime .nx not on the manifest must never leave Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked).Pros Source symbol found; stronger capability claim is authored; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading)Cons not adopted yet: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked) | ◉ | ○ | ○ | ◐ | ○ |
Public-release manifest for source and doctrine (which files are U//public-release, declared by the operator as data)DARK —runtime/nx_extllm.nx EXISTS but does not declare xl_release_manifest: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. The estate has a public-release flag on media vault items and a consent bit on documents, and NO marking on source; the manifest is that marking, and it is an operator decision row, never a defaultPros none measured yetCons open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ○ |
| TERMS | |||||
Provider terms ledger as data (retains, trains, output rights, benchmark-publish restriction, rate caps) read from the pinned mirrors and consumed by the guardSource present; authored stronger claim:xl_terms_ledger in runtime/nx_extllm.nx, verified at emit. The router publishes a data-policy field per model [oxalpha]; the contract binds to the EULA [stealtheula] and the terms [ortos] as rows, so a provider whose terms forbid a use refuses that use at the call, not in a reader's memory. The Claude escalation sink is in the ledger keyed on ACCOUNT TIER rather than vendor name, because the commercial contract [anthropic_comm_terms] and the consumer contract [anthropic_cons_terms] publish OPPOSITE training defaults, Claude Code inherits whichever account it runs under [anthropic_cc_datausage], and zero data retention is a per-organization grant that no plan name implies [anthropic_zdr] Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked).Pros Source symbol found; stronger capability claim is authored; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading)Cons not adopted yet: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked) | ◉ | ○ | ◐ | ○ | ○ |
| POOL | |||||
429-aware scheduler -- single lane per free pool, backoff from the measured refusal rate, concurrency pinned to what was admittedDARK —runtime/nx_extllm.nx EXISTS but does not declare xl_pool_sched: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. The gateway retries and falls back across providers [litellm]; ours schedules to the measured pool (2 of 6 admitted, ~50 permil-per-call refusal on first attempt, 2026-08-24) and never bypasses a limit, which the AUP forbids [stealtheula]Pros none measured yetCons behind LiteLLM gateway (leads), OpenRouter router (leads); open contract; expected source declaration not established | ○ | ◉ | ◉ | ○ | ○ |
| WORKER | |||||
Public-only context pack (rules corpus plus RELEASED signatures only, refusing any unreleased source)DARK —runtime/nx_forge_ctx.nx EXISTS but does not declare fc_emit_public: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. The variant of the pack an external worker may see; it composes the manifest and fails closed exactly as the existing pack fails closed on an UNVERIFIED rulePros none measured yetCons open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ○ |
External model backend for the forge with the same candidate contract as the local serveDARK —runtime/nx_forge_model.nx EXISTS but does not declare fm_post_ext: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. One more backend behind fm_compose_prompt and fm_strip_fences: the candidate lands in the same file the byte-exact judge reads, so nothing downstream changes [worker-mcp-exec-20260910]Pros none measured yetCons behind LiteLLM gateway, OpenRouter router, openai-evals (model grading); open contract; expected source declaration not established | ○ | ● | ● | ○ | ● |
Best-of-N sampling against the free verifier with the external tier as a samplerDARK —runtime/nx_forge.nx EXISTS but does not declare fg_best_of_n: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. Coverage climbs with samples given a verifier [llmonkeys]; ours is perfect and free, and a free external sampler makes N affordable -- the same contract /compare/coding names as CD2, so this cell and that one flip togetherPros none measured yetCons behind openai-evals (model grading); open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ● |
| JUDGE | |||||
External LLM-as-judge panel over U content, the independent referee grade recorded beside every claim it gradesOpen — watchingruntime/nx_llm_judge.nx : lj_grade, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. Model-graded evals are the field [openaievals]; the estate's law demands an independent referee grade with every progress claim -- free external judges over public artifacts are that referee, and this is the same contract the referee domain names [worker-evals-20260910]Pros none measured yetCons behind openai-evals (model grading) (leads); open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ◉ |
| PUBLISH | |||||
Publication guard -- external-model performance numbers stay C, capability rows stay USource present; authored stronger claim:xl_publish_guard in runtime/nx_extllm_publish_guard.nx, verified at emit. The Stealth AUP forbids publicly disseminating confidential technical information regarding the performance of the models [stealtheula]; the guard keeps the measured numbers on the internal plane and lets only the capability rows onto this page Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked).Pros Source symbol found; stronger capability claim is authored; ahead of LiteLLM gateway, OpenRouter router, LLM Guard (egress DLP), openai-evals (model grading)Cons not adopted yet: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked) | ◉ | ○ | ○ | ○ | ○ |
| LEDGER | |||||
Egress ledger row per call (provider, model, prompt hash, bytes, classification, terms pin, verdict)DARK —runtime/nx_extllm.nx EXISTS but does not declare xl_egress_ledger: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. Spend logs record cost [litellm]; ours records WHAT LEFT and under WHICH TERMS, so an audit of the lane is a plane read, and a terms change re-pins every row it governsPros none measured yetCons behind LiteLLM gateway, OpenRouter router; open contract; expected source declaration not established | ○ | ◐ | ◐ | ○ | ○ |
| WORKER | |||||
Rung worker loop -- reads a /compare plan rung whose spec and KAT are public, runs sampler then judge then bank, files the candidate for reviewDARK —runtime/nx_extllm.nx EXISTS but does not declare xl_rung_worker: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. The operator ask of 2026-08-24: free workers climb the open rungs, the gates rule, the assistant judges. Neg-control: a rung whose spec cites unreleased source must be SKIPPED by name, never prompted [worker-tools-20260910]Pros none measured yetCons open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ○ |
| DISTILL | |||||
Synthetic corpus harvest for the no-float LM, gated per provider on the output-rights row of the terms ledgerDARK —runtime/nx_extllm.nx EXISTS but does not declare xl_distill_corpus: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. The forge's compile-verified ledger is the training set the coding domain names; an external worker can grow it only where the provider's terms permit outputs to train a model [ortos] -- the ledger row decides, and UNKNOWN refusesPros none measured yetCons open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ○ |
| Class | Count | Note |
|---|---|---|
| rows with rival marks | 19 | rows carrying a Yes, Best or Part in a rival column |
| cited | 14 | a reference mark resolves to a pinned mirror |
| uncited | 5 | an observation read, badged until a reference lands |
| rows, summed | 19 | RECONCILED: the parts sum to the total 19 |
Delivery and evidence
Inspect risks, technical debt, rendered observations, experiments and references. Read scope and limitations alongside every result.
Debt register
| Id | Sev | What it is | Unblock |
|---|---|---|---|
| extllm-no-source-markingnot in this scope | 5 | The estate marks media vault items with a public-release flag and documents with a consent bit; runtime source carries no classification at all, so today a prompt built from a .nx file has no level to check. | XL2's release manifest is the marking for source; until it lands no external call may include any .nx content and the classifier does not exist to enforce that -- it is a written rule and nothing more. |
| extllm-key-exposed-not-vaultednot in this scope | 4 | The OpenRouter key was pasted into a chat on 2026-08-24, so it is exposed; it is not yet sealed in the vault and rotation is not armed. | Rotate the key at the provider, seal the replacement with the vault (vlt_derive), arm rot_due; XL3 reads only from the vault. |
| extllm-probe-numbers-internalnot in this scope | 3 | The 21-call probe of stealth/ox-alpha produced per-model performance numbers that the Stealth AUP forbids publishing [stealtheula]. | They stay on the internal plane at class C; XL8 is the mechanical guard; this page carries capability facts only. |
Risk register
| Risk | Likelihood x impact | Mitigation |
|---|---|---|
| A stub carrying a contract symbol flips a watch cell green with no capability behind it | possible x high | the flip is the receipt, the rung gate is the proof; every XL rung names its neg-control |
| The EULA and the model page disagree on training and a reader trusts the page | probable x high | XL1 reads the contract mirror, not the model card; the ledger row for ox-alpha carries trains_on_input YES with the EULA pin [stealtheula] |
| A provider's terms change under a pinned mirror | probable x medium | the refs gate re-pins RED on the next emit; XL1 refuses a row whose pin no longer matches until the mirror is re-fetched and re-read |
| Classification creep -- a C file is added to the release manifest to get work done | possible x high | manifest rows are operator-declared data, every call's ledger row carries the manifest pin, and the classifier's positive control is the public compare page, never a source file |
| The shared free pool starves the worker and a partial run reads as a failed one | probable x medium | XL3 attributes every 429 to its source and the worker loop appends each rung's result as it is decided, so an interrupted run keeps what it finished |
| Outputs harvested from a provider whose terms forbid training a competing model | possible x high | XL9 reads outputs_train_ours from the ledger and UNKNOWN refuses; Gemini unpaid, NVIDIA trial and Z.ai are worker or judge only by their own contracts [geminiterms] [nvidiatrial] [zaitos] |
Release history and work log
| When | Rung | Kind | Entry |
|---|---|---|---|
| 2026-09-10 | XL7 | research | 2026-09-10 operator supplied post-training/inference research added to candidate map, not implementation authorization. Primary checks: DeepSeekMath https://arxiv.org/abs/2402.03300 defines GRPO as PPO variant with group-relative rewards and reduced critic-memory requirements; it does not inherently optimize cleanest code. Phi4 https://arxiv.org/abs/2412.08905 credits data quality, curriculum and post-training with minimal architecture changes; task specialization need not remove general abilities. RLVR rewards only what its verifier covers; passing tests is not absence of defects. Immediate Beach worker investment stays inference-only: bounded task, context/tool contract, independent executable oracle, retained actual artifact and interaction evidence, resource/error measurements and calibrated escalation. First Fable public arithmetic fixture passed3 independent assertions, a calibration only; no general coding or deployment authority. No Nishi training dataset, fine-tuning or distillation initiated. New worker research refs in existing extllm.refs remain accessed-but-unpinned, not SOTA qualification. Continue comparative experiments across held-out real task families before raising worker authority. |
| 2026-09-10 | XL7 | measure | 2026-09-10 bounded supervised Claude usage baseline:2 completed Fable5.1 public calibration tasks on1 of3 authorized accounts (33.33% connections verified); work account excluded. Provider message IDs deduplicate3 assistant transcript rows to2 calls. Sum categories exactly once per message: uncached input4, cache creation13580, cache read66344, output863 including provider-reported thinking subset355; total token events80791. Shares: cache read82.12%, creation16.81%, output1.07%, uncached input0.005%. These are provider token events, not billable-dollar conversion or newly generated tokens. Both numeric fixture results independently checked;2/2 calibration outcomes is NOT general coding reliability. Zero Claude production deployments. Latest observed account allowance52%weekly and95%Fable used, usage credits off; percentages are account-specific provider observations, not estate task share. Nishi research3 attempted fetch invocations:2 actual source fetches succeeded with retained provenance,1 unsupported --help input failed; all must remain in accounting. Goal counter15372722 at its recorded observation is cumulative thread accounting, not a proven root-only/provider-normalized total. Cross-provider token/work percentages remain UNKNOWN until common time window, message deduplication, parent-child task attribution and executed/verified/delivered outcomes are joined. Existing Compare/model-worker and transcript-harvest owners must consume the evidence, not a new competing ledger. |
| 2026-09-10 | XL3 | land | 2026-09-10 PARTIAL MEASUREMENT CAPABILITY: existing nx_claude_harvest source adopted with predecessor retained; native60/60 regression checks passed and serving binary270544B SHA25610bf79c136ad63cfa1290279e1019541641bc5f9bb9baa9d602a51cc246e07d0 deployed by existing build lane. Direct API jobs1789052862/2863 distinguish missing-source exit3 from realempty exit0. Additive evidence_state and measurement_scope label observed tool frames; actor work share, token cost and verified delivery share remain null. Pending: shared read-integrity guarantees, response/message-ID token accounting, task-to-artifact acceptance joins, subscription dispatch and visible Compare regeneration. This does not complete XL3 or prove SOTA. |
| 2026-09-10 | XL3 | land | 2026-09-10 PARTIAL TEAM FOUNDATION: reused existing nx_estate_path.ep_anchor in actual nx_orchestrate_raci CLI and existing gate; no duplicate store or resolver. Read-only context probe4290 passed6/6: buildroot missing owners, serving root librarian/researcher resolved, unknown refused. Canonical gate4418 passed13/13 on live RACI and engagement store; emitted label corrected to resolve+plan+record, explicitly no workstream dispatch. Gate auto-refreshed146872B SHA3035a4b74da56b0a71c47da7c2d47e834d8aeca1d8365b1b6c5f2f574f7799ac. Actual CLI4419 build125896B SHAae42b09cf2f658d8d06bd81f6256c3e800ad04ac7d08dc86333a512a6d5f2862 planned organize successfully; no standalone deploy line, so standalone publication unverified. Missing bindings, evidence-write error propagation, account controls, advanced team lifecycle, portable configurable root and API registration remain open. Browser/video existing requirements remain scope; Beach agents continue private body and workspace integration. |
| 2026-09-10 | XL3 | land | 2026-09-10 PARTIAL TEAM OUTCOME INTEGRITY: actual nx_orchestrate_raci research run previously exited0 PLANNED with no workstream bound (4635). Corrected existing owner10367B, backup retained, adopted4735. Canonical build/run4744 now exits1 NOT-RUN-UNBOUND with reason/next action;126240B SHA96ee6f50791b771fd43dd5c7e94164a233978685ebfa53f1b6bbd9724592ea17. Private evidence-write failure control4708 used impossible directory under /dev/null, observed store lock open failure and exit1 evidence=write-failed without saved-record claim. Normal organize plan4715 remains exit0. Calls used sovereign build/API, no standalone dispatcher deployment line observed. These are failure-reporting and planning capabilities, not delivered autonomous research/advanced team execution; qualified workflow binding and native provider dispatch remain open. Beach page and body wiring continue in parallel. |
| 2026-09-10 | XL2 | measure | 2026-09-10 LIVE DELIVERY: native nx_extllm missing-ledger fix guarded-promoted via nx_organ_ship job1789073942, source16350B SHA8ba255bf46f585c1810b45ab736cbdc65958a0f6b7c20677e9764882a779ea53; live61520B SHA117a970f11b724a3fb2d853654facdcb1d8f2fe0112be97250313b046e09bebc. Pre-adoption actualCLI3/3 job3917 and content144 runs lost0 behaviorIDENTICAL3918 reviewed. Ship explicit gate3/3, matched liveSHA, refreshed offc twin, already registered; gate closure banking unproven direct-edge independence disclosed, URLverify skipped for CLI. Root then exercised registered live MCP/API:3960 missing journal exit3 evidence-unavailable with null counts;3961 actual empty exit0 zero;3964 populated fixture exit0 provider-filtered42bytes from140B ledger. This measures disclosed bytes, NOT provider token use. Generic short-read/malformed-ledger protection and native subscription dispatch remain incomplete. No provider call or paid API fallback occurred. |
| 2026-09-10 | XL2 | measure | 2026-09-10 PRIVATE HANDLER QUALIFICATION: existing ClaudeSubscriptionTasks owner now exposes typed status/start/result method handlers with host-injected authorization and included-Max-only policy. Root independently reran 24/24 checks including actual retained result and missing evidence, with provider calls disabled. Root caught valid directory without trailing separator failing; existing facade now normalizes it once, both forms tested. Source32133B SHA098a7c682dbf87bf544d15da2d7d9c23ea8f1e12b0d2d3b550469b9619ffb899. Evidence knowledge/gates/extllm-directory-normalization-20260910.json and extllm-typed-tools-qualification-20260910.json. Dedicated authenticated host registration remains OPEN; synthetic handler mounting is not native registration. No new service or paid provider fallback. Next integrate with existing host and retain actual provider dispatch receipts. |
| 2026-09-10 | XL2 | measure | 2026-09-10 ACTUAL SUBSCRIPTION DISPATCH: one public synthetic rational-clipping task ran on verified personal-aster Max, Opus5 selected after fresh Fable100percent-used refusal; extra usage disabled. Root read full prompt, provider result and review: exact crossing formula and three fixture answers accepted; collinear-degenerate handling and overflow-sizing guidance remain incomplete, not admitted as implementation. ModelUsage categories including ancillary Haiku:1279 input,3809 output,3284 cache-created,0 cache-read;8372 category sum is not subscription quota units. Provider estimate7000 under by1372 (19.6percent); actual root review tokens and net savings unknown. Fresh before/after weekly-used55percent unchanged at display precision, not proof zero consumption. List-price estimate is not billed spend. Evidence knowledge/gates/extllm-rational-clip-review-20260910.json;1 provider attempt,0 retries,0 deployments. Typed method call through Node MCP is verified; dedicated native host registration remains OPEN. |
| 2026-09-10 | XL2 | measure | 2026-09-10 TOKEN ROI INVESTMENT: operator prioritizes quality and timeliness, using included personal Claude Max capacity without paid API/overage fallback. Net root-token saving remains UNKNOWN: observed provider tokens are available for one latest reviewed task, but actual briefing/review/rework token attribution and a matched root-only counterfactual are absent. Do not turn unknown into zero or claim positive savings from completed responses. Existing delegation owner must retain task class, account/model identity, measured provider categories, actual supervision where observable, accepted/reworked/rejected outcome, integration evidence and elapsed time; counterfactual avoided effort must be labeled estimated with its comparison method. Rank reusable task classes by accepted integrated work with quality and delivery-time gates. Subscription quota percentages are separate from token counts; ancillary model usage remains attributed. No new provider dispatch or public Compare regeneration is established by this record. |
| 2026-09-10 | XL2 | measure | 2026-09-10 CONTEXT-INCLUSIVE ROOT USAGE: local current-root rollout token_usage_record exposes request response_id/thread_id/turn_id and usage categories. Independently deduplicated root window [21:48:00,21:52:00)UTC contains24 requests: input4279186, cached-input4229632 (subset), output4508, reasoning1216 (output subset), total4283694; noncached-input49554, cache-hit98.841976percent. All24 total=input+output and subset checks pass; zero duplicate responseIDs in window. This is mixed Beach/delegation/estate work, NOT dedicated Claude supervision; other Codex workers not included. Latest Claude calibration5167 reportedcategorysum accepted8/8 with no revision. No billed-cost, quota-conversion or net-savings claim. Retained request-level evidence C:/Users/elder/nishi-ops/root-token-window-20260910-2148-2152.json. Next existing usage owner ingests request records with explicit window/coverage and task-attribution evidence, keeping shared context cost visible and unknown attribution null. |
| 2026-09-10 | XL2 | measure | 2026-09-10 LOG STORAGE AND ARCHIVE AUDIT: metadata-only local scan measured Codex sessions1668829443B, nishi-ops1642700136B, .nishi9751126B; C available49379598336B of994834378752B. This is logical size in three directories, not full disk/NAS usage or causal attribution of disk pressure. Existing nx_jrnl_archive source has uncoordinated copy-then-truncate and overwrite-capable timestamp archive names; fsync results ignored. Existing nx_dr_archive caps source reads64MiB and can verify only that prefix. Existing retention policy is report-only and does not reclaim disk. Full source findings and dated official research retained in knowledge/gates/archive-source-audit-20260910-t132.json. No production archive, truncation or deletion executed. Repair existing owners; streaming immutable segments, checked durability, full restore verification and query continuity required before archival qualification. Compression alone does not bound retention; do not replicate full transcripts into accounting receipts. |
| 2026-09-10 | XL2 | invest | 2026-09-10 ROI direction: optimize accepted integrated capability per total root+worker investment, preserving or improving quality and delivery speed; spending more is acceptable for superior returns. Existing usage owner must separate request task/phase (implementation,research,review,retry,coordination,mixed-unknown) from input composition (new/repeated context,tool results,cached input) and output; overlapping categories never sum as disjoint spend. Rank native investment:1 artifact-backed compact tool results and reviewed-delta retrieval;2 scoped delegation packets with durable task state and recoverable workflows;3 measured prompt/retrieval/model-routing optimization. Full evidence remains retrievable; omitted failures, stale hashes, missing context and unknown coverage must fail acceptance. Research checked2026-09-10: RTK command-output reduction https://github.com/rtk-ai/rtk ; Pi runtime/state/telemetry https://github.com/earendil-works/pi ; DSPy GEPA metric-guided instruction optimization https://github.com/stanfordnlp/dspy/blob/main/docs/docs/diving-deeper/gepa-in-depth.md ; Microsoft LLMLingua https://github.com/microsoft/LLMLingua plus independent2026 evaluation https://arxiv.org/abs/2604.02985 . These are candidates, not measured Nishi savings or adopted dependencies. Benchmark matched Beach/swarm tasks on acceptance,missed defects,rework,wall time,root+worker tokens and resource use; retain setup cost and uncertain counterfactual. Lossy compression needs stronger exact-detail tests. Subscription-only/no paid-overage unchanged. Nishi search exact query context engineering returned unrelated engine results at top; coverage incomplete, so external primary-source research used. Reuse current catalogue/collector/dispatcher owners; no duplicate service or automatic training authorization. Public Compare UI regeneration not verified by this log. |
| 2026-09-10 | XL2 | invest | 2026-09-10 EVIDENCE-PRESERVING CAPABILITY ROI: operator requires lower token burden while intelligence,effectiveness and capability grow; details that reveal subtle defects must remain discoverable. Extend existing XL2 investment domain rather than new overlapping board. Compare raw-evidence baseline against proposed compact/delta workflow using retained cases: garment kind8 family4/5/8/9 selection divergence; mismatched shader source/layout; native usage conflicting duplicates and unknown evidence; whole-event leakage through local accounting normalization; persistent-state corruption. Acceptance requires catching the relevant defect with source-backed explanation, retrieving exact omitted bytes by immutable identity, and preserving missing/truncated/error markers. Measure root+worker tokens,coordination,retries,wall time,accepted integration,missed defects and one-time capability build cost separately; count cross-project reuse without double-counting outcome value. No compression percentage or lower model cost overrides quality/time requirements. Summarization is a navigation aid, never proof that unread source was reviewed. Current research/implementation remains partial; no optimality or public board-render claim. |
| 2026-09-10 | XL2 | measure | 2026-09-10 request accounting delivered through existing nx_claude_harvest usage-window <source.jsonl> <start-inclusive-epoch> <end-exclusive-epoch> <scan-start-byte> <scan-end-byte> and canonical local extllm_account_admission CodexRequestAccounting.observe. Native ship8654/call8688 verified44 requests7928230 tokens; direct original-log root24/4283694 and dispatcher20/3644536 match retained windows, canonical replay emits zero new requests/native calls. Adapter50202B SHAe8c68f57ac706d1b15f514df0348a451e5ea25a61df9872021a128820f365a51; existing checks owner preserves15 admission cases and reusable12 boundary cases pass. Whitelisted counters only, authenticated bounded cursor state, mixed evidence-linked phase annotations; billing/quota/task token partition/savings unknown. Receipt C:/Users/elder/nishi-ops/request-accounting-t76/canonical-local-delivery.json. Scheduler/UI, causal attribution, archive lifecycle and power-loss directory durability remain open. Research/optimization requirements reuse XL2 logs1789078983 and1789079136. |
| 2026-09-10 | XL2 | measure | Cross-estate capability ROI acceptance: compare matched-task baseline against all root, worker, review, retry and context effort; cached/new input/output remain orthogonal. Require quality, defect discovery, recovery and time-to-accepted-result gates. Exact raw evidence remains addressable with explicit omissions and uncertainty; missing usage is unknown. Conditional identity reader is first bounded intervention, not proof of preserved reasoning quality or ROI from smaller payload alone. Retained qualification compact-evidence-t77; private reader18/18 native, broader matched-task acceptance pending. |
| 2026-09-10 | XL2 | measure | Timestamp correction for preceding ROI acceptance row1789081960: its timestamp was clerical; actual append job1789081912 completed and was read back by observed2026-09-10T23:12:03Z. Acceptance content unchanged; prior record preserved. |
| 2026-09-10 | XL2 | measure | Confirmed native syscall correctness gap found during evidence-preserving reader qualification: nx_x86_64_ctx.nx x86ctx_rv64_to_x86_64_syscall line2252 maps RV64 pipe2(59) to x86 pipe(22), dropping NONBLOCK/CLOEXEC flags. Actual emitted buildroot/_build/nx_fsops_conditional_gate_t77.s sys_pipe2 sets rax22; job1789082125 timed out on intended nonblocking pipe write. Exact excerpt retained compact-evidence-t77/pipe2-emitted-mapping-refusal.txt. No compiler mutation; follow-up needs owning translation correction and real flags0/NONBLOCK/CLOEXEC, invalidflags, both-end F_GETFL/F_GETFD, and nonblocking-empty-read regression before compiler adoption. Current reader gate uses existing nx_fcntl_set_nonblock plus checked F_GETFL; this does not close pipe2 defect. |
| 2026-09-13 | XL3 | watch | 2026-09-13 DeepSeek trial preparation: native nx_extllm terms call verified job1789320866; nx_extllm_call registered (historical NO external call organ text is stale). Requested 4.1 identity, no-paid route and account data controls unverified; no inference, no harness install, no proprietary disclosure. Public synthetic cohort and total-cost unknowns retained in buildroot/knowledge/gates/deepseek-trial-preparation-t273.json. Hidden executable gate and blinded evaluation remain prerequisites, not delivered results. |
| 2026-09-13 | XL5 | land | 2026-09-13 Canonical nx_forge_judge now has eval-release for fixture and response JSON argument values (not file paths). Native helper gate16/16; canonical CLI1460 passed escaped equivalent key and preserves unknown build evidence; usage1480 returns expected2. Structured fixture/response SHA256 receipts, no substring acceptance for this command, never release authority. Callable through existing mgmt native qualification; named route registration remains unqualified. No provider calls or ROI claim. Evidence buildroot/knowledge/gates/forge-evidence-adoption-t274.json. |
Person · product · place — not yet measured for this domain
knowledge/compare/extllm.ppp (rows surface|nishi or c1..c4|label|url|connect naming OUR live surface and each rival's front door), run nx_ppp_probe domain extllm, and this section fills itself on the next beat: the same ruler on both sides — privacy and CX (third-party hosts, tracker classes, cookies, security headers), design and longevity (design hygiene, computed WCAG contrast, render-blocking resources, unsized media, script weight, theme and motion queries), findability (landmarks, skip link, on-site search, breadcrumb, headings, internal links).The field — discovered, not chosen
Rows written by nx_field_discover from extllm.seeds: the industry's own lists (Wikipedia wikitext, GitHub topics, awesome lists) read mechanically, every candidate counted across seeds. The matrix columns above are a SEAT'S pick; this band is the population they were picked from, and the stats line measures one against the other. A rival here is a lead, never a verdict — it earns a column when its capabilities are read and pinned.
| Class | Count | Note |
|---|---|---|
| seeds | 4 | |
| scanned | 4 | |
| fetched | 4 | |
| reused | 0 | |
| failed | 0 | |
| named | 4 | |
| candidates | 4 | |
| mentions | 4 | |
| capped | 0 |
| Rank | Rival | Seeds | Mentions | First seed | Kind | Link |
|---|---|---|---|---|---|---|
| 1 | LiteLLM gateway | 1 | 1 | col0-litellm | named | https://docs.litellm.ai/docs/ |
| 2 | OpenRouter router | 1 | 1 | col1-openrouter | named | https://openrouter.ai/terms/stealth |
| 3 | LLM Guard (egress DLP) | 1 | 1 | col2-llm | named | https://github.com/protectai/llm-guard |
| 4 | openai-evals (model grading) | 1 | 1 | col3-openai | named | https://github.com/openai/evals |
field candidates 4|shown 4 of 4|matrix columns in the field 4 of 4|discovered rivals with no column 0|malformed rows 0 (counted, never rendered)|read-capped 0
Exclusive and shared capabilities
This board grades 31 capabilities: 18 exclusive to it and 13 shared with other boards. Shared rows on this page: 13 = home here 2 + home elsewhere 2 + ambiguous 5 + undecided 4 + refused 0 (partition 13); undecided = unmapped 1 + unproven 3 + unevidenced 0; homonyms 0; since the previous census: new 0, resolved 0. Census FRESH age_s=6854.
| Capability | Organ | Boards grading it | Home | Rule | Evidence | Standing on this board |
|---|---|---|---|---|---|---|
| fc_emit | nx_forge_ctx.nx | coding, extllm, autodev | — | none | unproven | undecided |
| fg_run | nx_forge.nx | coding*, extllm, autodev | coding | rung | plan | home elsewhere adopt: @xrow coding|fg_run|home by rung (fleet duplicate census) |
| fg_bank | nx_forge.nx | coding*, extllm | coding | rung | plan | home elsewhere adopt: @xrow coding|fg_bank|home by rung (fleet duplicate census) |
| fg_best_of_n | nx_forge.nx | coding*, extllm*, autodev*, engineshift* | — | ambiguous | plan | ambiguous |
| lj_grade | nx_llm_judge.nx | referee*, extllm* | — | ambiguous | plan | ambiguous |
| gv_verdict | nx_gate_verdict.nx | codequality, extllm | — | none | unproven | undecided |
| rf_provenance | nx_research_fetch.nx | extllm, sovereignty | — | none | unmapped | undecided |
| sg_verdicts | nx_swebv_grade.nx | extllm, autodev | — | none | unproven | undecided |
| xl_call | nx_extllm_call.nx | extllm*, autodev*, engineshift | — | ambiguous | plan | ambiguous |
| xl_egress_class | nx_extllm.nx | extllm*, autodev, engineshift | extllm | rung | plan | home here |
| xl_terms_ledger | nx_extllm.nx | extllm*, engineshift | extllm | rung | plan | home here |
| xl_rung_worker | nx_extllm.nx | extllm*, engineshift* | — | ambiguous | plan | ambiguous |
| xl_distill_corpus | nx_extllm.nx | extllm*, autodev*, engineshift* | — | ambiguous | plan | ambiguous |
Gaps from the record — what the estate does that no board carries
The record census (nx_goalmap record) reads the invoked-tool population and every plan queue row and files each organ or directive that NO matrix, plan or gates row names. A row here is a callout the boards missed: adjudicate it onto a board or declare it infrastructure. Census state STALE (age 1395959 s), sources read 4 of 7 declared — a BLIND census is a FLOOR: unread sources can only add rows.
| kind | name | board | source | evidence |
|---|
rows shown 0|this board's directives 0|estate-wide un-boarded organs 452 (listed in full on /compare/ecosystem)|census rows 639|malformed 0 (counted, never rendered)
References
- [stealtheula] OpenRouter. Stealth Program End User License Agreement (updated 2026-07-06). The contract governing every stealth model: offered free of charge for purposes of collecting User Content for use in Stealth Model training and improvement; an irrevocable, perpetual, transferable licence to the provider; Exhibit A forbids publicly disseminating confidential technical information regarding the performance of the models and bypassing rate limits. publisher · read in our library
knowledge/fetched/cmp_extllm_stealtheula.html· pinhfed941390d8fbcb7b06d92337a4b367997e6de39452edec152dfee899f2f6a31· accessed 2026-08-24 · vendor-docGrounds: The TERMS ledger row, the PUBLISH guard row, the POOL scheduler row and the classification verdict: the prompt is the payment. - [oxalpha] OpenRouter. Ox Alpha model page (stealth/ox-alpha, released 2026-08-20). Free, 1,048,576-token context, 131,072 max completion, text plus image plus video in; the page banner states prompts and completions are retained by the provider and are not used for training, and defers all other use to the Stealth Model Terms. publisher · read in our library
knowledge/fetched/cmp_extllm_oxalpha.html· pinh108b8c85dfed1a9b5726af14f6ca955c1a7f5d56c3d6161f70009d88a42586e2· accessed 2026-08-24 · vendor-docGrounds: The OpenRouter router column and the subject of the 2026-08-24 probe; the banner-versus-contract conflict recorded in the verdict. - [ortos] OpenRouter. Terms of Service (last updated 2026-07-29). Input rights stay with the user; ownership of Output is set by each model's Model Terms; some models may store or train on Inputs; prompt logging is an opt-in; reselling API access or developing a competing service is forbidden. publisher · read in our library
knowledge/fetched/cmp_extllm_ortos.html· pinha502c47af66c96dc766085b59cf24e21ef8483a5446358ef09983b9b9e499780· accessed 2026-08-24 · vendor-docGrounds: The TERMS ledger row and the DISTILL row: output rights are per model, never assumed. - [orprivacy] OpenRouter. Privacy Policy (2026-07-06). OpenRouter does not use Inputs or Outputs for model training; image, audio and video are not persisted beyond routing; some Model Providers may use Inputs and Outputs for training or improvement. publisher · read in our library
knowledge/fetched/cmp_extllm_orprivacy.html· pinhc942b94888cc44af49f36d437b8f4f62788c278d6e47b4f591b4d175080e36db· accessed 2026-08-24 · vendor-docGrounds: The Market row: the router's own retention posture as distinct from each provider's. - [orlimits] OpenRouter. API reference, rate and daily limits. Free-model requests are capped per minute and per day, the daily cap rising with lifetime credits purchased; a negative balance returns 402 even on free models; capacity is governed globally. publisher · read in our library
knowledge/fetched/cmp_extllm_orlimits.html· pinh1bf49ef66aae54589791456d07c933a00a1e6df04fd2062066e040433b68ef49· accessed 2026-09-05 · vendor-docGrounds: Rung XL3 the scheduler and the Investment row: the caps the lane schedules to. - [orrouting] OpenRouter. Provider routing documentation. The per-request provider object accepts a data_collection allow or deny switch and a zero-data-retention flag, plus provider order, only and ignore lists; account settings can refuse routing to providers that train, with separate settings for paid and free models. publisher · read in our library
knowledge/fetched/cmp_extllm_orrouting.html· pinha6231076456cbde9053cd2a7bd3a67b9ee31c9b8f7755446bfa2eec2cdf11075· accessed 2026-09-05 · vendor-docGrounds: Rung XL3 and the small-business ladder rung: the router-side switch the call sets on every request. - [orfree] OpenRouter. Free models collection page. Lists the current free-suffixed models with their context lengths and carries provider data notices, including providers that state they may use free-tier inputs and outputs to train their models. publisher · read in our library
knowledge/fetched/cmp_extllm_orfree.html· pinh3cd5a178fdc63d69be9d84706e00fc7cb1540726a7720a87262d565803363953· accessed 2026-08-24 · mirrored-indexGrounds: The Market row: the free collection is provider-by-provider, never one policy. - [geminiterms] Google. Gemini API Additional Terms of Service (modified 2026-04-28). For Unpaid Services Google uses submitted content and generated responses to provide, improve and develop its products and machine-learning technologies; sensitive or confidential information must not be submitted; the Services may not be used to develop models that compete with them. publisher · read in our library
knowledge/fetched/cmp_extllm_geminiterms.html· pinhaf43b249eb59b519b7d7c8756b7f1e6a3c342d005354029222dff77c3c662858· accessed 2026-08-24 · vendor-docGrounds: Rung XL9: a provider whose contract forbids training a competing model stays worker or judge only. - [geminipricing] Google. Gemini API pricing page. The free tier rows are marked as used to improve Google products while paid rows are marked as not used; the free tier needs an active project, the first paid tier a linked billing account. publisher · read in our library
knowledge/fetched/cmp_extllm_geminipricing.html· pinh61e9a62971ad8b0dc74f8775c0f12d70d0408e94249186153c11655d64f80335· accessed 2026-08-24 · vendor-docGrounds: The Market row: the trains-on-data flag as the vendor itself publishes it. - [groqsa] Groq. Services Agreement (effective 2026-06-22). Groq does not access, use, store or retain Inputs or Outputs except as necessary to provide the service and is not permitted to use them for training or fine-tuning any models unless explicitly granted permission; the customer retains all intellectual-property rights in Inputs and Outputs. publisher · read in our library
knowledge/fetched/cmp_extllm_groqsa.html· pinh9fdcc9ed0cb4e7ae538233d4de66dfc43a247aefa0e24232ac05bf95963e8e2f· accessed 2026-08-24 · vendor-docGrounds: Rung XL9 and the research ladder rung: the first provider whose outputs may lawfully grow the forge ledger. - [groqlimits] Groq. Rate limits documentation. Per-model requests-per-minute, requests-per-day and tokens-per-minute rows for the free plan, published on the console. publisher · read in our library
knowledge/fetched/cmp_extllm_groqlimits.html· pinhb7499bf7162bd9282e43fcb600a733b6d41f913831b525913d27c62e96340da8· accessed 2026-08-24 · vendor-docGrounds: Rung XL3 and the Investment row: published caps the scheduler reads as data. - [cerebrastos] Cerebras. Terms of Service. Service content is not licensed to Cerebras for training or fine-tuning; Cerebras claims no ownership over Outputs; using the service for benchmarking or competitive analysis is prohibited; per the 2026-07-16 change the free tier became a card-gated trial. publisher · read in our library
knowledge/fetched/cmp_extllm_cerebrastos.html· pinh3bd7dfc787e80ef53e2b41dd3db852ba83967d6fec7885e8e4e12253fdb69a4d· accessed 2026-08-24 · vendor-docGrounds: Rung XL8 the publish guard and the Investment row: a benchmark-publish restriction as a ledger flag. - [sambanovaeula] SambaNova. Cloud End User License Agreement. Customer Content is excluded from the usage data collected; publishing or disclosing results relating to capabilities, performance or benchmarking without written consent is prohibited; the free tier applies when no payment method is linked. publisher · read in our library
knowledge/fetched/cmp_extllm_sambanovaeula.html· pinh9b6e8ccb70362e9194ae5f9b2521c193d0391259a9b2dc1f5c0fac7e0fd8ad92· accessed 2026-08-24 · vendor-docGrounds: Rung XL8: a second provider whose contract makes the publish guard mandatory. - [mistraltos] Mistral AI. Commercial Terms of Service (effective 2026-08-05). No training on Customer Data or Outputs except where the customer opted in, has not opted out of an opt-in-by-default product, gives feedback, or uses Labs or Preview models; the customer owns all Output. publisher · read in our library
knowledge/fetched/cmp_extllm_mistraltos.html· pinhc33421b30fcdf33a691130a9b922868a3e56d24afb4b0893a3547908de9135eb· accessed 2026-08-24 · vendor-docGrounds: The Market row: a conditional trains-on-data flag the ledger must carry per product, not per vendor. - [cfterms] Cloudflare. Service-Specific Terms, Developer Platform (2026-06-02). Unless otherwise agreed Cloudflare does not use Customer Content to train generative AI tools; the customer retains rights in Inputs and Outputs. publisher · read in our library
knowledge/fetched/cmp_extllm_cfterms.html· pinh7b60b139f3cfac0fe326e8f14ee7091a32bbd1b494e39741fb8afd95cf31d3e4· accessed 2026-08-24 · vendor-docGrounds: Rung XL9 and the market-leader ladder rung: a no-training contract with outputs retained. - [cfpricing] Cloudflare. Workers AI pricing. A daily allowance of neurons at no charge, with some models requiring a paid billing method. publisher · read in our library
knowledge/fetched/cmp_extllm_cfpricing.html· pinh8275a38368b69821aa56ca21e5489d1fc319cab1696180b61c270fa39d447c6d· accessed 2026-08-24 · vendor-docGrounds: The Investment row: the free allowance as published. - [nvidiatrial] NVIDIA. API Trial Terms of Service (version 2025-09-19, PDF). Access is for limited trial purposes and not production; NVIDIA collects User Content and Generated Content to improve its products and services including AI models; the API service including Generated Content may not be used to develop or improve competing products. publisher · read in our library
knowledge/fetched/cmp_extllm_nvidiatrial.pdf· pinhafd5df0322615ff95736f2f8b59fa952f5f4582dfe05f97213c0ae860f628b30· accessed 2026-08-24 · vendor-docGrounds: Rung XL9: trial-only and competing-use clauses keep this provider worker or judge only. - [zaitos] Z.ai. Terms of Use (2026-04-14). End User Content is not used for developing or improving the Services without explicit agreement; utilising platform models or exported data to develop, train or enhance competing algorithms or models is prohibited; the Flash models are free. publisher · read in our library
knowledge/fetched/cmp_extllm_zaitos.html· pinhe14e9cae5a6fb0177687bed9d4a453341ba89c3d8219f43b10851f6ebe88ce21· accessed 2026-08-24 · vendor-docGrounds: Rung XL9 and the Investment row: free and not trained on, but outputs may not train a competing model. - [deepseektos] DeepSeek. Open Platform Terms of Service (2026-04-29). Inputs and Outputs may be applied to derivative product development and to training other models such as model distillation; rights in Outputs are assigned to the user; the service is prepaid. publisher · read in our library
knowledge/fetched/cmp_extllm_deepseektos.html· pinh2b433c53cbac75491959025eba5a27908f9f1886352dce0a05f3d5a936d87790· accessed 2026-08-24 · vendor-docGrounds: Rung XL9 and the Buy-vs-build row: the one contract that permits distillation by name. - [alibabaquota] Alibaba Cloud. Model Studio new-user free quota. Each model carries an independent free quota, typically one million tokens, valid for ninety days. publisher · read in our library
knowledge/fetched/cmp_extllm_alibabaquota.html· pinhb5842e49cf1d0e10422ffa2bbcf87025e6990b18c6c8f2a76aeffd0cc3a6eaa6· accessed 2026-08-24 · vendor-docGrounds: The Market row: a per-model quota rather than a rate cap. - [alibabaprivacy] Alibaba Cloud. Model Studio privacy notice. States that customer data is never used for model training. publisher · read in our library
knowledge/fetched/cmp_extllm_alibabaprivacy.html· pinh70b9453fa2d16ad627fd6d72313c23b5ecaeb2fcef69ce7c2cef7f4303ae446d· accessed 2026-08-24 · vendor-docGrounds: The Market row: the trains-on-input flag for this provider; its output-rights row stays UNKNOWN until the service agreement is mirrored. - [ghmodels] GitHub. GitHub Models documentation. States that as of 2026-07-30 GitHub Models has been fully retired. publisher · read in our library
knowledge/fetched/cmp_extllm_ghmodels.html· pinh6decf8c242b2657407e64a04c31220801ac19f6222186ce77d16e58779ff6fcd· accessed 2026-08-24 · vendor-docGrounds: The Market row: a free tier that no longer exists, recorded so nobody re-discovers it. - [anthropic_comm_terms] Anthropic. Commercial Terms of Service effective 2025-06-17. Section B states that Customer retains all rights to its Inputs and owns its Outputs, that Anthropic assigns to Customer its right title and interest in Outputs, and that Anthropic may not train models on Customer Content from Services. Section D.4 forbids using the Services to build a competing product or to train competing AI models. publisher · read in our library
knowledge/fetched/cmp_extllm_anthropic_comm_terms.html· pinhb19a3c3a15f3aa5896ba7e2e6eca0c88f7d92c16613dd78ec2b2f659372eec97· accessed 2026-08-25 · vendor-docGrounds: The TERMS ledger row for the Claude escalation sink and the reason DISTILL is refused on every Anthropic tier. - [anthropic_cons_terms] Anthropic. Consumer Terms of Service effective 2025-10-08. The use-of-Materials clause states that Anthropic may use Materials to improve the Services and to develop other products including training our models unless the user opts out through account settings, and that even after opting out Materials are used for training when the user provides Feedback or when content is flagged for safety review. publisher · read in our library
knowledge/fetched/cmp_extllm_anthropic_cons_terms.html· pinh584de77005454c3cbd6dbddf85be507dd32dbfd1c032133dd65b16bc426534de· accessed 2026-08-25 · vendor-docGrounds: The TERMS ledger row proving the consumer default is the OPPOSITE of the commercial default, so the ledger must key on account tier and never on vendor name. - [anthropic_cc_datausage] Anthropic. Claude Code documentation Data usage page. States that new models are trained using data from Free Pro and Max accounts when the model improvement setting is on including when Claude Code is used from those accounts, that commercial users are not trained on unless they opt in through a programme such as the Development Partner Program, and gives retention as five years consumer opted-in, thirty days consumer opted-out and thirty days commercial standard. publisher · read in our library
knowledge/fetched/cmp_extllm_anthropic_cc_datausage.html· pinh09ef84d3ee4e3e32516b85efc770d322fba094707a5c57f2a6b50ad1abf350f3· accessed 2026-08-25 · vendor-docGrounds: The TERMS ledger row for the Claude tier as actually operated: Claude Code has no data posture of its own and inherits the signed-in account tier. - [anthropic_zdr] Anthropic Privacy Center. Zero data retention article dated 2026-06-09. States that zero data retention is granted per organization subject to Anthropic approval, applies to eligible APIs and to Claude Code accessed via a commercial organization API key, that User Safety classifier results are retained regardless, and that it is not included in the standard Enterprise plan. publisher · read in our library
knowledge/fetched/cmp_extllm_anthropic_zdr.html· pinh4d6e3b7ccfb410005afdf8e8f2a515f31c4a1aadac77a2921b2bddc8fbe32feb· accessed 2026-08-25 · vendor-docGrounds: The TERMS ledger retention column: the only published posture that reduces retention below the commercial default, and it cannot be assumed from a plan name. - [litellm] BerriAI. LiteLLM documentation. The open-source gateway that routes one OpenAI-shaped call across many providers with key management, retries, fallbacks and spend logging per key. publisher · read in our library
knowledge/fetched/cmp_extllm_litellm.html· pinh7907d422518d59baa217e3a41966ae74e67b0fad493e7ee82e3239ef2ec9e987· accessed 2026-08-24 · vendor-docGrounds: The LiteLLM gateway column: routing, keys and spend as the field's shape. - [llmguard] Protect AI. LLM Guard repository. An open-source input and output scanner for LLM prompts: secrets, PII and injection patterns, redaction and refusal. publisher · read in our library
knowledge/fetched/cmp_extllm_llmguard.html· pinh91f889d011d0a925a9d7ecddbddc81ed877648ecdfe6ccc2d0b43a40022fb8f0· accessed 2026-08-24 · source-readGrounds: The LLM Guard egress DLP column: a pattern list where the estate has a clearance level. - [openaievals] OpenAI. Evals repository. The evaluation framework and registry, including model-graded evals in which a model grades another model's output. publisher · read in our library
knowledge/fetched/cmp_extllm_openaievals.html· pinhca1f4b2e7e7d80f566b098eb751c9362bc708fc2757146efbf98d7cb22634ae1· accessed 2026-08-24 · source-readGrounds: The openai-evals column and rung XL6: model-graded evaluation as the field's judge. - [llmonkeys] Brown, Juravsky, Ehrlich, Clark, Le, Re and Mirhoseini. Large Language Monkeys: Scaling Inference Compute with Repeated Sampling (arXiv 2407.21787, 2024). Coverage climbs with the number of samples when a verifier can pick the correct one. publisher · read in our library
knowledge/fetched/cmp_extllm_llmonkeys.html· pinh922be22acaa0999ba5690c50a680d29228db6c0e73aa6c89ffb769acb7cab68c· accessed 2026-08-24 · published-paperGrounds: Rung XL5 best-of-N and the innovation ladder rung: the result that makes a free sampler worth scheduling. - [swebench] SWE-bench. The public leaderboard and task set for repository-level issue resolution. publisher · read in our library
knowledge/fetched/cmp_extllm_swebench.html· pinhf70fc6c0c500d07af0e7e3d7d7f3a51c40689a7f1704eebf4702b8cacc114411· accessed 2026-08-24 · mirrored-indexGrounds: The BENCH row and the research ladder rung: the repository-level bar the estate's own grader scores against. - [worker-tools-20260910] Anthropic. Writing effective tools for agents (2025-09-11), revisited2026-09-10. Evaluate representative tasks, tool errors, tokens and latency; compare held-out tasks and consolidate repeated useful workflows. publisher · accessed 2026-09-10 · vendor-docGrounds: Candidate improvements to existing worker tools; not evidence of smaller-model Nishi parity.
- [worker-evals-20260910] Anthropic. Demystifying evals for AI agents (2026-01-09), accessed2026-09-10. Agent evaluation must inspect outcomes and traces across repeated trials. publisher · accessed 2026-09-10 · vendor-docGrounds: Independent grading and task-specific worker qualification; a single arithmetic pass confers no general authority.
- [worker-mcp-exec-20260910] Anthropic. Code execution with MCP (2025-11-04), accessed2026-09-10. Programmatic composition can move intermediate processing outside model context; requires measured effectiveness and execution controls. publisher · accessed 2026-09-10 · vendor-docGrounds: Reuse existing API composition and tool discovery owners; no automatic efficiency claim or replacement of estate APIs.
generated by nx_swcompare_matrix (sovereign NishiLang organ) from knowledge/compare/extllm.matrix · source checks show implementation presence; runtime and user-outcome evidence are reported separately · JavaScript supports page controls