code wiki / _hdl_build / nx_referee.nx

nx_referee.nx

buildroot/runtime/_hdl_build/nx_referee.nx

3229 B63 linesdepth 2pulls 2 transitivereach 11 importersview sourcekind librarytopic referee
docsdependenciesstructsconstsfunctions

about

nx_referee.nx -- the NISHI REFEREE / ADJUDICATOR (operator: "we seem to always get biased testing"; the racing role was scoped for running heats, but SCORING them fairly needs a separate, unbiased judge). The race in nx_race_vs_claude used CLAUDE'S OWN self-assessed accuracies -- the competitor graded itself, which is the exact bias to kill. The Referee enforces the bedrock of every fair judge system (code-golf / ICPC / Codeforces / ML leaderboards), bits-up: 1. SEPARATION -- the scorer is NOT a competitor (no self-grading) 2. GROUND-TRUTH -- score against a known-correct ORACLE, not opinion 3. HIDDEN HOLDOUT-- the test cases were not seen/tuned-on by the competitor (anti-overfit/Goodhart) 4. DETERMINISM -- same submission -> same score, every run (no LLM-judge variance, no flaky harness) 5. DECISIVENESS -- declare a winner ONLY if the margin beats a noise threshold; else TIE (no overclaim) A head-to-head is VALID only if all fairness gates pass. license_tier: ORIGINAL NOTE v1: core principles; to be EXTENDED by the running deep-research (contamination detection, adversarial/secret test generation, special-judge checkers, statistical-significance tests).

dependencies 1 imports · 6 importers

nx_syscalls.nx nx_referee.nx nx_paper_panel.nx nx_referee_serve.nx nx_referee_test.nx nx_referee_v2.nx nx_referee_v2_test.nx nx_research_exceed.nx

imports: nx_syscalls.nx

imported by: nx_paper_panel.nxnx_referee_serve.nxnx_referee_test.nxnx_referee_v2.nxnx_referee_v2_test.nxnx_research_exceed.nx

structs

none

consts

17const REF_TIE: i64 = 0
18const REF_A_WINS: i64 = 1
19const REF_B_WINS: i64 = 2

functions

23func ref_score(outputs: *i64, oracle: *i64, n: i64) -> i64
called by 2: mainmain
29func ref_abs(x: i64) -> i64 { if x < 0 { return 0 - x } return x }
32func ref_decisive(score_a: i64, score_b: i64, margin_threshold: i64) -> i64
called by 1: ref_verdict calls 1: ref_abs
38func ref_verdict(score_a: i64, score_b: i64, margin_threshold: i64) -> i64
called by 2: mainmain calls 1: ref_decisive
47func ref_no_self_grade(scorer_id: i64, competitor_a_id: i64, competitor_b_id: i64) -> i64
54func ref_hidden(seen_by_competitor: i64) -> i64 { if seen_by_competitor == 0 { return 1 } return 0 }
called by 3: mainmainmain
57func ref_test_valid(no_self_grade: i64, hidden: i64, has_oracle: i64, deterministic: i64) -> i64
called by 3: mainmainmain