nishi code wiki / research / character body physics

Character body physics, animation and secondary dynamics

SOTA census · compiled 2026-08-01 · 5 axes · ~38 sourced claims · 8 declared gaps

lineage: forked from research_rtrender parent domain: embodiment

Claims are labelled SHIPPING / RESEARCH / ANNOUNCED / ABANDONED. Gaps are declared UNVERIFIED rather than guessed. Numbers marked [computed] or [synthesis] are derived estimates, not measurements.

HEADLINE — nothing on the visible path requires runtime ML, and the policy was never the bottleneck anyway. A DeepMimic/AMP-class controller is ≈0.8M params ≈ 1.6 MFLOP per inference — at 30 Hz control that is ~50 MFLOP/s per NPC, well under 5% of one core with WASM SIMD int8 [computed] — but every one of these policies is trained inside an Isaac-Gym/MuJoCo-grade float simulator at 120 Hz+ substeps, so the real risk for an integer engine is the sim-to-sim gap, not inference. Meanwhile no shipped title uses a learned physics policy as primary character control: the production recipe in 2025–26 is motion matching + partial ragdoll blending + IK, and fixed-point rigid-body physics is already production-proven (Photon Quantum, Q48.16, shipped in Stumble Guys and LEGO Brawls).

1. Learned physics control — the lineage and the honest WASM verdict

ModelScale / costWhat it addedStatusSource (date)
DeepMimicMLP (1024, 512) hidden ≈ 0.8–1.5M params; 30 Hz control over a 1.2 kHz simPer-clip RL motion trackingRESEARCHarxiv.org/abs/1804.02717 (2018)
AMPSame MLP scaleAdversarial motion prior replaces the tracking rewardRESEARCHarxiv.org/abs/2104.02180 (2021)
PHCProgressive multiplicative control policy (new capacity per hard-motion cluster)One controller imitates ~all of AMASS + fall recoveryRESEARCHgithub.com/ZhengyiLuo/PHC (ICCV 2023)
PULSEDistils PHC into a 32-dim latent motion spaceDistillation is load-bearing: 97.1% AMASS-test imitation with distillation vs 32.6% pure-RLRESEARCHarxiv.org/abs/2310.04582 (ICLR 2024)
MaskedMimicTransformer-VAE, tens of M paramsUnified control as masked motion inpainting — driven by any subset of {keyframes, joints, text, objects}RESEARCHarxiv.org/abs/2409.14393; dl.acm.org/doi/10.1145/3687951 (SIGGRAPH Asia 2024)
SONICTracking scaled 1.2M → 42M params, 100M+ frames / 700 h mocap, 21k GPU-h; policy inference 1–2 ms, kinematic planner 12 ms (NVIDIA hardware, unspecified)The “foundation controller” moment; one token space serves VR teleop + VLARESEARCHarxiv.org/abs/2511.07820; nvlabs.github.io/GEAR-SONIC/ (2025-11)

Training lives in NVIDIA ProtoMotions, the now-standard open framework (SHIPPING SDK, nvlabs.github.io/ProtoMotions/user_guide/experiments.html, 2024–25). The 2025–26 successor wave: PDP — expert RL trackers behaviour-cloned into one diffusion policy (dl.acm.org/doi/full/10.1145/3680528.3687683, SIGGRAPH Asia 2024); SuperPADL — progressive supervised distillation to 5000+ language-directed skills, real-time on a consumer GPU (arxiv.org/pdf/2407.10481, 2024); CLoSD — a real-time autoregressive motion-diffusion planner (as few as 10 diffusion steps) closed-loop with an RL tracker (arxiv.org/abs/2410.03441, ICLR 2025 spotlight); UniPhys (arxiv.org/pdf/2504.12540, 2025-04); PARC — a self-bootstrapping generator↔tracker data flywheel for parkour (dl.acm.org/doi/10.1145/3721238.3730616, SIGGRAPH 2025); BFMTrack (arxiv.org/pdf/2606.25056, 2026-06); InterPrior (arxiv.org/pdf/2602.06035, 2026-02); MaskAdapt (arxiv.org/pdf/2603.29272, 2026-03); and NVIDIA's 21 SIGGRAPH 2026 papers including MotionBricks, one neural model animating game characters and driving a Unitree G1 robot (xenospectrum.com/en/nvidia-siggraph-2026-physical-ai/, 2026-07). Graphics↔humanoid-robotics convergence is now the field's centre of gravity.

What runs at 60 fps in WASM [computed]: an AMP/DeepMimic/DReCon-class policy (obs≈200 → 1024 → 512 → act≈30) is ≈0.8M params ≈ 1.6 MFLOP/inference; SONIC's smallest tier (1.2M) is the same class — that fits, with int8 quantisation, at <5% of one core per NPC at 30 Hz control. MaskedMimic-class transformers (tens of M params + a text encoder) are not 60 fps-WASM material. The policy is not the bottleneck; the physics sim it was trained in is — see the cross-cutting risks below.

2. What shipping games actually use (2025–26)

ThingStatusSource (date)
Motion matching — UE 5.4 production-ready; “battle-tested in Fortnite Battle Royale… shipped on all platforms from mobile to console, running on all 100 player characters plus NPCs”SHIPPINGunrealengine.com/en-US/blog/unreal-engine-5-4-is-now-available (2024-04); lineage: For Honor gdcvault.com/play/1023280 (GDC 2016), TLOU2 (2020)
Euphoria-class active ragdoll — a Rockstar monopoly; no other 2025/26 AAA ships full behavioural active ragdollSHIPPINGgta.fandom.com/wiki/Euphoria; gtaboom.com former-Rockstar-animator interview (2025, secondary)
GTA VI — Dynamic-Motion-Synthesis-lineage stack rebuilt for current genANNOUNCEDsame gtaboom.com interview (2025); release 2026-11-19
skate. — physics-based skating + signature ragdoll as core gameplay (EA Full Circle, F2P, PS/Xbox/PC)SHIPPINGnews.ea.com press release (early access 2025-09-16)
DReCon — RL policy tracks motion-matching output on a simulated ragdoll, “high responsiveness at low runtime cost”RESEARCHubisoft.com La Forge (SIGGRAPH Asia 2019)
Learned Motion Matching — NN-compressed MM databaseRESEARCHdl.acm.org/doi/abs/10.1145/3386569.3392440 (TOG 2020)
ML-assisted motion matching in Far Cry 6 toolingSHIPPING (tooling)80.lv/articles/ubisoft-explained-ai-driven-motion-matching-technique-used-in-far-cry-6 (2021)
EA patent: periodic-autoencoder motion alignment — ML animation entering AAA pipelinesGRANTEDUSPTO 12403400 (2025)

Takeaway: no shipped title yet uses a learned physics policy as primary character control. The shipped recipe is motion matching + partial ragdoll blending + IK. Motion matching itself is a kNN search over pose+trajectory feature vectors — no floats required at runtime, no ML; the real cost is mocap curation. See also “Ragdoll Motion Matching”, GDC ML Summit (gdcvault.com/play/1026712).

3. Simulation methods on a tiny integer budget

MethodStatusSource (date)
XPBD — compliance-based constraints, timestep-independent stiffness; substeps beat solver iterations (60 Hz frame → 2–4 substeps × 1 iteration)RESEARCH, widely adoptedmatthias-research.github.io/pages/tenMinutePhysics/09-xpbd.pdf (2016/2022)
VBD — vertex-level Gauss-Seidel block descent on implicit Euler; unconditionally stable, massively parallelRESEARCHarxiv.org/pdf/2403.06321 (TOG 2024)
AVBD — augmented Lagrangian → hard constraints: rigid stacking, friction, articulated bodies with joint limits, stiff-soft coupling; shown live at Real-Time Live! (“Crazy Fast Physics”)RESEARCHgraphics.cs.utah.edu/research/projects/avbd/; dl.acm.org/doi/10.1145/3731195 (SIGGRAPH 2025-08)
Featherstone — reduced coordinates, O(n), zero joint drift for a ~20-DOF ragdoll; what MuJoCo/Isaac useRESEARCH, classicstandard texts; harder in fixed point (mass-matrix conditioning, trig) — overkill vs maximal-coordinate XPBD+joints for game NPCs
Photon Quantum — full deterministic physics on Q48.16 fixed point (FPVector/FPQuaternion); shipped Stumble Guys, LEGO BrawlsSHIPPINGdoc.photonengine.com/quantum/v2/quantum-intro; blog.photonengine.com/photon-quantum-is-now-free-for-development/ (2024–25)
BEPUphysics1int (C# fixed-point 3D) · SG Physics 2D (Godot)SHIPPING SDKgithub.com/sam-vdp/bepuphysics1int; gitlab.com/snopek-games/sg-physics-2d (n.d., repos)
Practical recipe [synthesis]: simulate 12–20 physics bodies per NPC — not the 104 render joints; map bodies to major joints and keep fingers/face kinematic. XPBD spheres/capsules, joint constraints with limits plus motor constraints pulling toward the animation pose (PD-in-position form), 60 Hz with 2 substeps. Budget ≈ 20 bodies × ~30 constraints ≈ low tens of µs per NPC in scalar integer WASM [computed]. Adopt AVBD's augmented-Lagrangian hard joints if XPBD drifts. Photon Quantum's Q48.16 proves the arithmetic regime in production; nx_collide2d's swept-integer discipline is the same family.

4. Cloth, hair and flesh secondary dynamics

ThingStatusSource (date)
Cloth — XPBD remains the real-time default; VBD beats PBD in throughput/stability in a Unity head-to-headRESEARCHmdpi.com/2076-3417/14/23/11072 (2024)
Strand hair — id/MachineGames shipped strand-based hair as the ONLY human hair solution, at 60 Hz cross-platform, in Indiana Jones and the Great Circle; full GPU raster/shading pipelineSHIPPINGadvances.realtimerendering.com/s2025 “Strand Hair in IJGC” slides (2025-08)
Neuralocks — real-time neural hair simulationRESEARCHarxiv.org/pdf/2507.05191 (2025-07)
Spring-Decomposed Skinning — real-time secondary jiggle/follow-through from helper spring bones on the rig; no neural net; a direct fit to a 104-joint LBS engineRESEARCHonlinelibrary.wiley.com/doi/10.1111/cgf.70209 (CGF 2025)
Musculoskeletal-driven skin deformation NN · Hierarchical Neural SkinningRESEARCHdl.acm.org/doi/10.1145/3658135 (TOG 2024); dl.acm.org/doi/10.1145/3728300 (2025-05)
UE 5.6 Chaos Flesh — tet-based muscle sim in a real-time engineANNOUNCEDdredyson.com (2025 — secondary source, flagged UNVERIFIED below)
FreeMusco — motion-free latent control for morphology-adaptive musculoskeletal locomotionRESEARCHarxiv.org/pdf/2511.14205 (2025-11)

Verdict: full muscle simulation does not transfer to this budget. The pattern that does is offline sim → cheap runtime approximation — bake muscle/flesh behaviour into joint-driven secondary dynamics (spring bones, ML-deformer pattern). The low-budget hair transfer is dozens of guide strands as Verlet/XPBD chains + skinned interpolation — a nx_softdyn-shaped problem.

5. What this means for the Nishi stack

Constraint filter: proprietary language compiled to WASM · integer/fixed-point math core · WebGL2 · no third-party physics engine (no PhysX/Havok/Jolt) · reimplementation from papers only.

Substrate in handState
NXA skinned bodies — 14,164 verts / 28,092 tris / 104 joints / 17.9 s mocap, linear blend skinning in the vertex shaderlive
nx_softdyn — general spring-damper secondary-dynamics solvergate-proven 7/7, not yet bound to any geometry — nothing visibly jiggles
nx_collide2d — swept continuous 2D collider8/8, mutation-proven
RTX 5080 16GBoffline training only — never a runtime dependency
#TechniquePayoff / costExit criterion
1Spring-bone secondary dynamics on the existing rig (SDS pattern, CGF 2025)Jiggle, inertia, follow-through, hit-twitch via helper joints driven by nx_softdyn. Days; zero new solver; fixed-point trivial. Bodies stop looking dead.nx_softdyn bound to ≥8 helper joints on the NXA body; visible follow-through in the 17.9 s mocap replay at 60 fps; replay bit-identical across two runs.
2Motion matching for NPC locomotionProven on mobile-class hardware for 100+ characters (Fortnite/UE 5.4). kNN over integer pose+trajectory features — no floats, no runtime ML. Real cost = mocap curation.Live segment selection from the mocap DB; transition pose-error under a declared integer threshold; ≥10 NPCs at 60 fps in WASM.
3Passive XPBD ragdoll, fixed pointDeaths, falls, impacts. 12–20 bodies, capsules, joint limits, 2 substeps @ 60 Hz. Photon Quantum Q48.16 proves the regime. Weeks on top of softdyn's PBD core.Ragdoll drops, settles, and stays settled: joint limits hold, no overflow, bit-exact determinism gate across runs.
4Active ragdoll = ragdoll + motor constraints toward the animation pose + hand-authored stumble/brace/grab/get-up state machine, blended per body partThe Euphoria-class MVP and the literal “NPCs are real body physics” milestone — no ML required. Adopt AVBD hard joints if XPBD drifts.A character pushed mid-walk recovers or falls believably and gets up; per-body-part blend live; 60 fps sustained.
5Utility-AI daily-life layer (NPC-minds scope, listed for build order)Needs/schedules/affordances + event memory, all integer scores; inZOI's ~600-datapoint pattern shows the ceiling with zero runtime LLM (nvidia.com ACE, 2025-03).Hundreds of NPCs run coherent daily schedules for a full sim-day with zero runtime ML.
6DReCon-class learned tracking policyRobust balance/push-recovery that rung 4 can't hand-author. Train AMP-style ~1M-param MLP on the RTX 5080 against a headless build of THE fixed-point sim (or PDP-style BC on in-engine rollouts); quantise int8.Policy tracks mocap in-engine under perturbation; <1 ms/NPC at 30 Hz control in WASM, measured (not computed).
7LLM NPC dialogue/replanning (NPC-minds scope)Minitron-class 0.5–2B, server-side or WebGPU worker; the LLM writes/edits rung-5 schedules, never ticks the sim.Free-text goal edit changes an NPC's schedule with the render loop unaffected.
8Distilled foundation controller (MaskedMimic/SONIC → SuperPADL/PDP-style student)Text/keyframe-conditioned full-body skills in one small policy. Endgame; field trajectory (SONIC 1.2M tier, MotionBricks) says this commoditises via ProtoMotions checkpoints by ~2027.Only after rungs 3+4+6 are green and the training loop against the in-house sim is proven; one policy serves ≥3 distinct skill families in-engine.
Two cross-cutting risks, stated plainly. (a) Every learned-control paper assumes a float simulator — DeepMimic through SONIC all train in Isaac-Gym/MuJoCo-grade float sims with accurate contacts at 120 Hz+ substeps. A fixed-point engine cannot consume their checkpoints or their training recipes as-is; it needs its own training-time twin (a headless native build of the same integer sim) or PDP-style behaviour cloning on rollouts collected in-engine. (b) 104 render joints is NOT the physics skeleton. Keep physics bodies at 12–20, map them to major joints, retarget to the render rig, and keep fingers/face kinematic — every production ragdoll works this way. Corollary: rungs 1–4 require zero runtime ML and already deliver the visible goal.

Declared UNVERIFIED — do not treat as measured

  1. All WASM throughput figures ([computed]: 1.6 MFLOP/inference, <5% of a core, low tens of µs/NPC XPBD) are arithmetic estimates. No int8 MLP or integer XPBD benchmark has been run in the Nishi WASM runtime.
  2. SONIC's 1–2 ms policy inference was reported without the GPU model; it is not evidence about CPU/WASM latency.
  3. UE 5.6 Chaos Flesh rests on a secondary source (dredyson.com); Epic release notes were not checked.
  4. The GTA VI Euphoria-lineage rebuild rests on one ex-Rockstar animator interview (gtaboom.com, 2025); Rockstar has not confirmed the stack.
  5. Exact parameter counts for PHC and MaskedMimic were not captured — treat “progressive capacity” and “tens of M” as order-of-magnitude.
  6. Whether any 2025/26 title beyond skate. and GTA VI ships active ragdoll as a core mechanic was not swept.
  7. In-production use of DReCon / Learned Motion Matching beyond Ubisoft's own publications: no independent evidence either way.
  8. BEPUphysics1int and SG Physics 2D shipped-title lists were not verified; they are cited as fixed-point precedent code, not adoption evidence.

Method. Primary source: a dedicated research agent's July/August 2026 web sweep across six question axes (learned control, shipping practice, simulation methods, cloth/hair/flesh, NPC minds, adoption ladder); its claims, numbers and URLs are carried verbatim here. The agent's task-output file was written as 0 bytes by the harness; the full report was recovered from its JSONL transcript. No fresh URL-by-URL re-verification pass was run for this brief — anything that failed the sweep's own sourcing bar is listed above rather than smoothed over. New measurements in the Nishi WASM runtime (item 1) are what would most change the conclusions.