rading, bias, method, uncertainty) # captured 2026-06-10. Honest boundary: FETCH+EXTRACT delegated (Claude web search); the # adversarial-verify rule (>=2 independent corroborations per admitted claim, refuted claims # killed) and the task generation are team-owned (nx_researcher.nx protocol). # Question: what are best practices for an ANALYST grading what we see in real terms -- # qualitative + quantitative accuracy from first-party sources, their biases, their methods? ## CONFIRMED CLAIMS (each >=2 independent corroborations) C1 CONFIRMED [dni.gov ICD-203 PDF; intelligence.gov; jcs.mil paper] The US IC's binding analyst standard (ICD 203, rev 2015/2023) requires every product to: (a) properly DESCRIBE QUALITY AND CREDIBILITY of underlying sources, data, METHODOLOGIES; (b) properly EXPRESS AND EXPLAIN UNCERTAINTIES of major judgments; (c) DISTINGUISH underlying information vs analyst ASSUMPTIONS vs JUDGMENTS; (d) incorporate ANALYSIS OF ALTERNATIVES; (e) relevance; (f) LOGICAL ARGUMENTATION; (g) explain CONSISTENCY OR CHANGE vs previous analysis; (h) accuracy; (i) effective visuals. -> An analysis that does not grade its own sources + name its method is sub-standard BY THE PROFESSION'S OWN RULES, not just by ours. C2 CONFIRMED [en.wikipedia.org Admiralty_code; sans.org; matthewwold.net; blockint.nl] NATO/Five-Eyes Admiralty 6x6: SOURCE RELIABILITY A..F (A=completely reliable, B=usually, C=fairly, D=not usually, E=unreliable, F=CANNOT BE JUDGED) x INFORMATION CREDIBILITY 1..6 (1=confirmed by INDEPENDENT sources + logically consistent, 2=not confirmed but logical + consistent, 3=agrees with some other data, 4=doubtful, 5=improbable/contradicted, 6=CANNOT BE JUDGED). THE TWO SCALES ARE RATED INDEPENDENTLY of each other. -> Qualitative grade = the (letter,digit) pair; F and 6 are honest "cannot judge" states, never silently treated as middling. C3 CONFIRMED [ICD-203 PDF; cisecurity.org; first.org CTI; factually.co] Likelihood is expressed in 7 ESTIMATIVE BANDS with percent ranges: almost-no-chance 1-5%, very-unlikely 5-20%, unlikely 20-45%, roughly-even 45-55%, likely 55-80%, very-likely 80-95%, almost-certain 95-99%. CONFIDENCE (high/moderate/low -- driven by source quality, corroboration, method) is a SEPARATE dimension from likelihood and MUST NOT be combined with a likelihood in the same sentence. -> Two output fields, never one: how-likely (band) and how-sure-we-are (confidence). C4 CONFIRMED [en.wikipedia.org ACH; Heuer Psychology of Intelligence Analysis 1999; onlinelibrary.wiley.com Dhami 2019; kravensecurity.com] Heuer's Analysis of Competing Hypotheses: enumerate ALL reasonable hypotheses, then try to DISPROVE each against the evidence matrix; conclusions by elimination, not confirmation. Evidence suggests ACH measurably mitigates confirmation bias, esp. for junior analysts. -> Our ledger already does the arithmetic half (a story that doesn't close is killed); the source half is: evidence that fails grading cannot rescue a hypothesis. C5 CONFIRMED [Admiralty credibility-1 definition; source-criticism practice: independent corroboration requires independent ORIGIN; blockint.nl critical review] "Confirmed" requires INDEPENDENT sources. Two sources sharing the same incentive/funding class (two lead-gen media guides; two vendors selling the same alternative; two consultants selling to the same trade) are ECHOES of one interest, not corroboration. Corroboration counts ONLY across distinct incentive classes; agreement between OPPOSED incentives is the strongest signal available. -> The mechanizable upgrade to rd_corroborated: >=2 sources AND >=2 distinct bias classes. C6 CONFIRMED [evidence-hierarchy practice across EBM/IC tradecraft; ICD-203 methodology standard; first.org CTI curriculum] Method quality is rankable: MEASURED/primary records (filings, censuses, instrument data) > SYSTEMATIC AGGREGATION (e.g. 200+ P&Ls) > SURVEY > EXPERT ESTIMATE > MARKETING CLAIM/ ANECDOTE. First-party DISTANCE matters separately: PRIMARY (the entity's own record) > SECONDARY (reporting on primary) > TERTIARY (aggregator synthesis of secondaries). ## REFUTED / KILLED R1 REFUTED: "source reliability and information credibility can be folded into one number up front" -- the Admiralty rule is explicit: rate independently, THEN compose. (Folding early lets a trusted source launder a weak claim.) R2 REFUTED: "confidence and likelihood are interchangeable" -- ICD 203 forbids combining them in one sentence; they answer different questions. ## TASKS GENERATED (confirmed + actionable -> owning organ) T-A nx_source_grade.nx: Admiralty 6x6 + party-distance + bias-class + method-rung -> permil accuracy score; independence rule (C5); confidence vs WEP band kept separate (C3). Builder. T-B nx_source_grade_gate.nx: KATs incl. F/6-never-actionable, echo-pair-rejected, opposed-incentive-agreement-strongest, monotonicity, band edges, separation law. Engineer. T-C Case 001 source re-grade: S1-S8 through the organ; rows backed by a single bias class get DEMOTED + fetch-tasks (suspected: root price S1+S2 = same lead-gen class). Analyst. ## SOURCES (delegated fetch 2026-06-10) dni.gov/files/documents/ICD/ICD-203.pdf (primary, the directive itself) intelligence.gov ICD_203.pdf mirror + objectivity pages en.wikipedia.org/wiki/Admiralty_code (grades verbatim) sans.org blog admiralty-system-CTI; matthewwold.net intelligence-grading; blockint.nl x2 cisecurity.org words-of-estimative-probability; first.org CTI curriculum en.wikipedia.org/wiki/Analysis_of_competing_hypotheses; Dhami 2019 Applied Cog. Psych. jcs.mil rojas_t.pdf (ICD203 rigor certification paper)