code wiki / (root) / nx_benchmark_harness.nx

nx_benchmark_harness.nx

buildroot/runtime/nx_benchmark_harness.nx

13259 B348 linesdepth 2pulls 2 transitivereach 4 importersview sourcekind benchtopic benchmark
docsdependenciesstructsconstsfunctions

about

nx_benchmark_harness.nx -- head-to-head perf verdict primitive. Substrate-level enforcer of the cardinal feedback-honest-perf- verdict-no-aspirational-claims. Every NishiLang vs incumbent benchmark emits a SEALED-ENUM verdict per axis: WIN / LOSE / TIE / UNMEASURABLE. No "competitive with" weasel words. Every LOSE row MUST be paired with a named-improvement string the caller fills in. IDEA-PROVENANCE (per the language-design-research discipline: learn from, never copy): - JMH (OpenJDK) -- multi-warmup, multi-fork, GC-quiet - Criterion.rs -- bootstrap CI, outlier detection (MAD) - Google Benchmark -- statistical-rigor patterns - Hyperfine -- multi-trial CLI command timing Every idea was re-derived from the published papers + project READMEs; no source code was incorporated. See genealogy_id for paper citations. Six measurable axes per benchmark run: 1. THROUGHPUT ops/sec or items/sec 2. LATENCY_P50 median single-op time 3. LATENCY_P99 99th-percentile single-op time (tail latency) 4. MEMORY_PEAK peak bytes during run 5. DETERMINISM same-input-same-output across N trials (Q10) 6. PORTABILITY runs without modification across target archs (Q10) Each axis emits a per-axis sealed-enum verdict: NX_BENCH_AXIS_WIN challenger >= incumbent * 1.01 (1% delta floor) NX_BENCH_AXIS_TIE within 1% either direction NX_BENCH_AXIS_LOSE challenger < incumbent * 0.99 NX_BENCH_AXIS_UNMEASURABLE measurement not available this run Composite verdict: NX_BENCH_DECISIVE_WIN >= 4 of 6 axes WIN, zero LOSE NX_BENCH_WIN majority WIN, fewer LOSE NX_BENCH_TIE majority TIE or balanced WIN/LOSE NX_BENCH_LOSE majority LOSE NX_BENCH_DECISIVE_LOSE >= 4 of 6 axes LOSE NX_BENCH_INCONCLUSIVE too many UNMEASURABLE to call Why 1% floor: per cardinal feedback-one-percent-minimum-delta-substrate

dependencies 2 imports · 4 importers

nx_syscalls.nx nx_tier.nx nx_benchmark_harness.nx nx_benchmark_harness_test.nx nx_daemon_bench.nx nx_daemon_bench_test.nx nx_polyglot_bench.nx

imports: nx_syscalls.nxnx_tier.nx

imported by: nx_benchmark_harness_test.nxnx_daemon_bench.nxnx_daemon_bench_test.nxnx_polyglot_bench.nx

structs

92struct AxisMeasurement
99struct AxisVerdict
105struct BenchmarkReport

consts

58const NX_BENCH_Q: nx_int = 1024
61const NX_BENCH_AXIS_UNMEASURABLE: nx_int = 0
62const NX_BENCH_AXIS_LOSE: nx_int = 1
63const NX_BENCH_AXIS_TIE: nx_int = 2
64const NX_BENCH_AXIS_WIN: nx_int = 3
65const NX_BENCH_N_AXIS_VERDICTS: nx_int = 4
68const NX_BENCH_AXIS_THROUGHPUT: nx_int = 0
69const NX_BENCH_AXIS_LATENCY_P50: nx_int = 1
70const NX_BENCH_AXIS_LATENCY_P99: nx_int = 2
71const NX_BENCH_AXIS_MEMORY_PEAK: nx_int = 3
72const NX_BENCH_AXIS_DETERMINISM: nx_int = 4
73const NX_BENCH_AXIS_PORTABILITY: nx_int = 5
74const NX_BENCH_N_AXES: nx_int = 6
77const NX_BENCH_INCONCLUSIVE: nx_int = 0
78const NX_BENCH_DECISIVE_LOSE: nx_int = 1
79const NX_BENCH_LOSE: nx_int = 2
80const NX_BENCH_TIE: nx_int = 3
81const NX_BENCH_WIN: nx_int = 4
82const NX_BENCH_DECISIVE_WIN: nx_int = 5
83const NX_BENCH_N_COMPOSITE_VERDICTS: nx_int = 6
86const NX_BENCH_DELTA_FLOOR_Q10: nx_int = 10
90const NX_BENCH_MIN_TRIALS: nx_int = 8

functions

129func _bench_axis_verdict(m: *AxisMeasurement, v: *AxisVerdict) -> nx_int
177func _bench_composite(report: *BenchmarkReport) -> nx_int
256func _bench_worst_axis(report: *BenchmarkReport) -> nx_int
306func nx_benchmark_compute(report: *BenchmarkReport) -> nx_int
320func nx_benchmark_axis_verdict_is_valid(v: nx_int) -> nx_int
called by 1: main
326func nx_benchmark_composite_is_valid(c: nx_int) -> nx_int
called by 1: main
332func nx_benchmark_axis_index_is_valid(a: nx_int) -> nx_int
342func nx_benchmark_report_is_cardinal_compliant(report: *BenchmarkReport) -> nx_int
called by 1: main