nx_codegen_bench_gate.nx
buildroot/runtime/nx_codegen_bench_gate.nx
about
nx_codegen_bench_gate.nx -- OBJECTIVE code-gen grader (eats the anti-navel-gazing debt, 2026-07-15).
nx_capability_triage graded code-gen = MEASURE(bench=none): we had NO objective way to grade generated
code, so any "our coding is improving" claim was navel-gazing. This is the missing instrument: coding
tasks with OBJECTIVE (input->expected) test cases; a candidate solution (a function = "generated code")
is run against every case; PASS iff ALL cases match. The grade is not our opinion -- it is whether the
code COMPUTES THE RIGHT ANSWER. The load-bearing ANTI-NAVEL-GAZING property (T2): the grader must FAIL
wrong code -- a grader that passes broken solutions is exactly the navel-gazing the operator warned about.
T1 CORRECT solutions PASS (the tasks are gradable; reference code scores 100%)
T2 SOUND / anti-navel-gazing: BROKEN solutions FAIL (the grader rejects wrong code -- the whole point)
T3 SCORE computed = pass-rate over candidates (the metric), persisted to a durable bench ledger (living)
T4 DETERMINISM: re-grading is byte-stable
The candidate here is a compiled-in function (proves the GRADER sound); the real GENERATOR (forge e1 coder
-> candidate SOURCE -> nx_cc compile -> this grader) + a standard suite (HumanEval-style) = the next rungs
that turn this into the external number proving "we reduce Claude coding". expect_exit: 0 license_tier: ORIGINAL
dependencies 2 imports · 0 importers
imports: nx_syscalls.nxnx_gate_verdict.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| none |
functions
| 18 | func cb_w(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 } |
| 19 | func cb_n(v: i64) -> i64 |
| 34 | func t1_correct(a: i64, b: i64) -> i64 { if a > b { return a } return b } |
| 35 | func t1_broken(a: i64, b: i64) -> i64 { return a } // always first arg -- WRONG |
| 37 | func t2_correct(n: i64, unused: i64) -> i64 { var s: i64=0; var i: i64=1; while i<=n { s=s+i; i=i+1 } return s } |
| 38 | func t2_broken(n: i64, unused: i64) -> i64 { return n } // just returns n -- WRONG |
| 40 | func t3_correct(n: i64, unused: i64) -> i64 { if n < 2 { return 0 } var d: i64=2; while d*d<=n { if (n/d)*d==n { return 0 } d=d+1 } return 1 } |
| 41 | func t3_broken(n: i64, unused: i64) -> i64 { return 1 } // everything prime -- WRONG |
| 44 | func cb_grade(cand: func(i64,i64) -> i64, ins1: *i64, ins2: *i64, exp: *i64, ncase: i64) -> i64 called by 1: main |
| 54 | func main() -> i64 |