Experiential Testing
Target (state of the art): All 19 census axes PRESENT with measured head-to-head gates vs the SOTA tool of each axis (Playwright/Lighthouse/axe-core/BackstopJS class), every capability wired as: sovereign organ + logical API + MCP tool + workstream row + agent/RACI owner, driving BOTH endpoints (Nishi browser native; third-party browsers via oracle drivers) on desktop AND mobile. The census page /experiential is EMITTED by the census run and can never drift from disk truth.
The ladder — how we get there
| status | rung | the number it moves | proof |
|---|---|---|---|
| DONE | screenshot capture + scroll recording (nishi_gui nxshot lane) | census axis: GAP -> PARITY (disk-fact) | recordings published: /evidence/browser-parity-2026-07-16b; wiring next: MCP tool nx_browser_shot |
| DONE | visual diff / regression (nx_visual_diff: grid parity + content extent + worst-cells) | census axis: GAP -> PARITY; the Chrome-parity benchmark's ruler | gate 4/4 (identical=100%, half=50%, oracle-self=100% ceiling); wiring next: MCP read tool + CI sweep caller |
| DONE | automated page / asset verify (nx_page_verify) | every published page gated (assets fetch AND decode) | in production across /evidence + /experiential ships; wiring: already the ship-lane gate |
| DONE | UI / CX quality judging (nx_uxcx_grade + nx_ux_study NN/g method) | census axis PARITY; study grades bound to task outcomes, sticker-proof | nx_ux_study_gate 4/4 with neg-controls; PASS earned on /experiential readability |
| DONE | live-run intelligence / findings (nx_ux_study findings: severity + heuristic + frame evidence + recommendation) | census axis: GAP -> PARITY | every finding frame-grounded; drove 5 root fixes 2026-07-16 |
| DONE | nx_browser_drive v1 — script-driven click/scroll/open/wait/shot on the Nishi browser (the census keystone) | census: drive-browser + input-drive + auto-wait GAP -> PARITY (14 GAP -> 11) | DRIVE-COLLECT GREEN; interaction recording /evidence/browser-drive-2026-07-16 (clicked -> navigated the live ladder) |
| NEXT | drive v2: type + form fields; CDP oracle driver for third-party browsers | input-drive axis toward full parity; cross-browser oracle axis | task 6 continuation |
| QUEUED | network interception + console/error capture organs | their two census axes (kept honestly GAP in v1 — separate organs by design) | nx_net_intercept + nx_console_capture |
| NEXT | CI / sweep orchestration (nx_experiential_sweep: page-list x surface-list -> census+study+diff per target, cron'd) | census axis GAP -> PARITY; the "consistent repeatable" engine that re-runs everything and re-emits all pages | wiring: workstream row + hostctl cron + MCP trigger |
| QUEUED | accessibility audit (nx_a11y_audit: WCAG contrast/alt/landmark checks on our DOM) | census axis GAP -> PARITY vs axe-core | oracle = axe-core corpus already fetched |
| QUEUED | performance audit (nx_perf_audit: paint-time, bytes, request counts vs web-vitals class) | census axis GAP -> PARITY vs Lighthouse | paint-time instrumentation exists in the shot lane |
| QUEUED | agentic goal-directed play (nx_playtest_agent: goal -> drive loop -> findings; the DOTA-style playtesting) | census axis GAP -> PARITY vs browser-use | depends on nx_browser_drive; evidence = play recordings |
| QUEUED | video / media playtest (play the sovereign /video client + third-party media, judge frames) | census axis GAP -> PARITY | rides the codec lane + drive |
| QUEUED | trace / replay (record fetch+layout+paint+input timeline, replay deterministically) | census axis GAP -> PARITY vs Playwright traces | recording lane is the seed |
| QUEUED | true video encode of recordings (frames -> .nxv via sovereign codec on the /video client) | evidence recordings become playable video, not frame-steppers | codec client already live at /video |
| QUEUED | mobile / device emulation (viewport+DPR+touch profiles in the Nishi browser; CDP mobile for oracles) | census axis GAP -> PARITY | the browser already takes viewport args in the shot lane |
| QUEUED | head-to-head EXCEED gates (per axis: our organ vs the tool, measured) | the only path any axis can ever read AHEAD (census caps PRESENT at PARITY by design) | pattern: nx_capability_census h2h slot already in the schema |
Runs — the ratchet, dated
- 2026-07-16 — KEYSTONE v1 LIVE: script drove the Nishi browser (click -> navigated the live ladder page, recorded); census GAP 14 -> 11, PARITY 8
Live surfaces
- /experiential — The live census (EMITTED by nx_s21_census — cannot drift from disk truth)