code wiki / _hdl_build / nx_referee.nx
nx_referee.nx
buildroot/runtime/_hdl_build/nx_referee.nx
about
nx_referee.nx -- the NISHI REFEREE / ADJUDICATOR (operator: "we seem to always get biased testing";
the racing role was scoped for running heats, but SCORING them fairly needs a separate, unbiased
judge). The race in nx_race_vs_claude used CLAUDE'S OWN self-assessed accuracies -- the competitor
graded itself, which is the exact bias to kill. The Referee enforces the bedrock of every fair judge
system (code-golf / ICPC / Codeforces / ML leaderboards), bits-up:
1. SEPARATION -- the scorer is NOT a competitor (no self-grading)
2. GROUND-TRUTH -- score against a known-correct ORACLE, not opinion
3. HIDDEN HOLDOUT-- the test cases were not seen/tuned-on by the competitor (anti-overfit/Goodhart)
4. DETERMINISM -- same submission -> same score, every run (no LLM-judge variance, no flaky harness)
5. DECISIVENESS -- declare a winner ONLY if the margin beats a noise threshold; else TIE (no overclaim)
A head-to-head is VALID only if all fairness gates pass. license_tier: ORIGINAL
NOTE v1: core principles; to be EXTENDED by the running deep-research (contamination detection,
adversarial/secret test generation, special-judge checkers, statistical-significance tests).
dependencies 1 imports · 6 importers
imports: nx_syscalls.nx
imported by: nx_paper_panel.nxnx_referee_serve.nxnx_referee_test.nxnx_referee_v2.nxnx_referee_v2_test.nxnx_research_exceed.nx
structs
| none |
consts
| 17 | const REF_TIE: i64 = 0 |
| 18 | const REF_A_WINS: i64 = 1 |
| 19 | const REF_B_WINS: i64 = 2 |
functions
| 23 | func ref_score(outputs: *i64, oracle: *i64, n: i64) -> i64 |
| 29 | func ref_abs(x: i64) -> i64 { if x < 0 { return 0 - x } return x } |
| 32 | func ref_decisive(score_a: i64, score_b: i64, margin_threshold: i64) -> i64 |
| 38 | func ref_verdict(score_a: i64, score_b: i64, margin_threshold: i64) -> i64 |
| 47 | func ref_no_self_grade(scorer_id: i64, competitor_a_id: i64, competitor_b_id: i64) -> i64 |
| 57 | func ref_test_valid(no_self_grade: i64, hidden: i64, has_oracle: i64, deterministic: i64) -> i64 |