nishi code wiki / research / realism oracles

Perceptual realism oracles: what can actually be gated per build

Published 2026-08-02 · lineage: forks off the craft SOTA roadmap's ruler problem (§4, "our critic saturates at 1000 and we score 983") and extends the feel-oracle idiom already live in knowledge/gamefeel_oracle.conf · sibling of the generational-SOTA method brief

The question this brief answers. We can already measure our own physics against cited biomechanics bands. What can we measure about perceived realism — automatically, on every build, without a human in the loop? The honest answer splits three ways: image fidelity is gateable today with published pass lines; motion physical-plausibility is gateable with closed-form arithmetic; and presence and "does this character look alive" have no validated automated metric at all. Knowing which third a question lives in is the whole value here.

1. Image fidelity — gateable today, with real numbers

The decisive 2025 result is that metrics trained on compression artifacts do not transfer to rendering artifacts. CGVQM (Intel Labs) was calibrated on exactly our failure classes — path-tracing sample counts, neural denoising, neural supersampling, Gaussian splatting, frame interpolation, variable-rate shading — over 4,800 subjective ratings, and it beats every general-purpose metric by a wide margin.

Metric on rendered content (CGVQD dataset)	PLCC	SRCC
PSNR	0.697	0.723
FLIP	0.693	0.728
LPIPS	0.696	0.748
VMAF	0.715	0.790
ColorVideoVDP	0.744	0.775
CGVQM-5	0.871	0.877

The pass lines worth adopting, each derived from a primary source rather than taste:

Gate	PASS	Where the number comes from
CGVQM-2/5 vs golden replay	≥ 87.5 / 100	the shipped code maps 0–100 onto the ITU-R BT.500 five-grade scale; `round(q/25)==4` is the "imperceptible" grade
ColorVideoVDP	≥ 9.68 JOD (≤1 JND); strict ≥ 9.84	inverting the JPEG AIC2026 conversion `JND = 3.1889·(10−JOD)^1.0129`
Fidelity vs a path-traced reference	10·log₁₀(rmsDistanceRatio) ≤ −22 dB	Google `<model-viewer>` render-fidelity-tools — RMS colour distance ≤ 0.63% of maximum, normalised over model pixels so background cannot dilute it

Sources: arXiv:2506.11546 (CGVQM, 13 Jun 2025) · IntelLabs/cgvqm, Apache-2.0 · arXiv:2509.13150 (near-JND metric evaluation, 16 Sep 2025) · google/model-viewer render-fidelity-tools, Apache-2.0

Two traps. FLIP has no published pass threshold — it was designed for error maps, so anyone gating on mean FLIP invented the number. And LPIPS is the wrong tool for a "did this get subtly worse" gate: in the near-JND band the learning-based metrics performed poorly because their training data never covered high-fidelity differences.

2. Motion plausibility — closed-form arithmetic, no model downloads

These need only joint positions and a mass model, run in milliseconds, and have ground-truth values published from real motion capture. They are the cheapest high-value oracles in this brief.

Gate	PASS	Note
Foot-skate ratio (slide >2.5 cm while foot <5 cm high)	≤ 0.060, alert if < 0.03	real mocap measures 0.057 — this is a closer-to-ground-truth-is-better metric; driving it to zero produces robotically pinned feet
Ground penetration (5 mm tolerance)	= 0.000	from the PhysDiff triple, now de facto standard
Foot clipping (feet interpenetrating, 5 cm)	= 0.000
PFC (physical foot contact)	≤ ~1.15× own ground-truth corpus	does not assume feet should be static, so it survives sliding motion honestly

Sources: arXiv:2212.02500 (PhysDiff, ICCV 2023) · arXiv:2310.08580 (OmniControl, ICLR 2024) · arXiv:2410.07296 (ReinDiffuse, WACV 2025) · Stanford-TML/EDGE (PFC), MIT

Do not gate on motion FID. It has saturated (0.045 versus a real-motion 0.002 after three years of architecture work), swings 2.5× purely from swapping feature extractors, shows no sample-level correlation with human judgement, and a controlled study injecting foot-skate into ground truth found a Fréchet motion distance cannot detect skating at all — the single most common animation artifact.

3. The cross-cutting law: pretty is not physical

The Physics-IQ benchmark measured r = −0.46 between visual realism and physical understanding across video generators — the model judged most visually realistic scored worst on physics. Beauty and physics must be gated on separate axes, or you ship a gorgeous character who moonwalks. This is the strongest argument for keeping our physics oracle (biomechanics bands) independent from any future image-fidelity oracle.

Source: arXiv:2501.09038 (Physics-IQ, Jan 2025) · google-deepmind/physics-IQ-benchmark

4. Frame budgets and what a browser can actually observe

Non-negotiable: 72 Hz = 13.9 ms, 90 Hz = 11.1 ms, 120 Hz = 8.3 ms, with an explicit rule that any JavaScript taking longer than two milliseconds should be optimised. In-browser, the only honest dropped-frame detector is the delta between consecutive animation-frame callbacks against the nominal rate, reported as p50/p95/p99 — motion-to-photon latency, reprojection events and compositor timing are simply not exposed to the web platform, and the request for those statistics has been open since 2021.

Sources: Meta WebXR performance workflow · immersive-web/webxr#1203 (open since 2021-06-16)

A frame-time gate measured under software rendering is meaningless. Headless Chromium needs --enable-unsafe-swiftshader, and then the renderer string reads "Google SwiftShader" — those frame times say nothing about shipping performance. Scope such a gate to algorithmic-complexity regression only, and say so. This directly constrains our own review lane, which runs exactly that way.

5. How to shape the harness (steal these, not the thresholds)

Skia Gold's shape: content-hash digests, baselines outside version control, and several approved images per configuration — enumerate legitimate outputs rather than loosening tolerance until noise passes.
Sobel-masked comparison: zero tolerance everywhere except detected edges. The highest-leverage trick for surviving anti-aliasing differences between machines.
Blender's loosening doctrine: one strict reference platform; relax the failing-pixel-count budget before per-pixel tolerance; never degrade everything to the worst platform's noise floor.
Auto-tune thresholds against a corpus of known-good and known-bad frames instead of hand-picking them.

Sources: Chromium GPU pixel testing with Gold · gold.skia.org

UNVERIFIED / declared gaps

Presence has no automated metric. The best published result (93% accuracy) requires EEG and skin-conductance sensors on a live person, its code is unreleased, and it regresses onto questionnaires that a 2025 study found could not detect an embodiment manipulation at all. Not immature — nonexistent.
"Does this character look alive" has no validated automated metric. Every validated uncanny-valley instrument is a human questionnaire whose items are adjective pairs. One 2025 attempt at image-feature prediction exists on 8,099 frames with no independent validation and no confirmed release.
Cybersickness-from-video is not usable software. The strongest released-and-reproducible signal is a rank correlation of about 0.665 — enough to order two builds, not to state a build's sickness score. Two leading repositories are empty or their dataset host is dead.
No judder score, no FLIP threshold, no SSIM pass line, no standardised real-time renderer fidelity benchmark, and no published convergence criterion for declaring a path-traced image "the reference" exist anywhere in the literature. Whatever we build here is ahead of the published state of the art, which also means we must calibrate our own numbers rather than cite someone else's.
Not yet measured by us: every threshold in this brief is adopted from primary sources; none has yet been run against our own renderer. The next rung is wiring tier-1 rows into the existing feel-oracle table and recording the first measurements.
License landmines flagged and avoided: FovVideoVDP is non-commercial, the widely-used image-quality toolbox is PolyForm Noncommercial, a popular lip-sync metric ships no licence file, and two motion critics are copyleft or unlicensed. The metrics recommended above are Apache-2.0, MIT, BSD or equivalent.