nishi code wiki / research / social interaction
Social & physical NPC interaction: approach, conversation, contact
Published · lineage: forks off NPC intelligence (minds, advertisements, gossip) and character physics (the motion substrate) · context: making a beach full of SITUATIONS — the player, a woman, or a woman and her friends approach and engage in conversation and physical interaction.
Freshness declared up front. This session's web-search budget was exhausted before this brief was written, so every claim below is verified only to the author's May-2026 knowledge horizon plus the sovereign corpus; the May→July 2026 tail is UNVERIFIED and said so, not guessed. A re-verification pass (search +
nx_page_ingest per citation) is the declared next act for this page, per process rule A. Items most likely to have moved in that tail are flagged inline.The three findings that shape the build. (1) The moment-to-moment social loop in every SHIPPING life sim is still utility scoring over advertised interactions — the same smart-object architecture our engine already runs; LLM agents ship only in narrow, latency-tolerant slots. (2) The academically correct frame for “a woman and her friends approach and form a conversation” has existed since 1990: F-formations — agents arrange into an o-space ring with defined entry positions; proxemics gives the distances. It is cheap integer math, not ML. (3) Two-character PHYSICAL interaction is the hard part everywhere: research generates it with diffusion/transformers offline, but everything that SHIPS uses paired, pre-authored clips anchored to a shared alignment frame with contact IK fixups — which is exactly what an integer, no-compute-shader engine should build.
1. What ships today: the social-sim architecture bar
| System | Status | What it proves |
|---|---|---|
| The Sims 4 interaction system (EA/Maxis, 2014–live) | SHIPPING | Interactions are data rows on objects and people (advertise → score by need/relationship → queue); social menus are just the scored list made visible. Multitasking = interleaved interaction queues. The durable lesson: a person is a smart object. |
| inZOI (Krafton, early access 2025-03-28) | SHIPPING (EA) | The current visual bar for life sims (UE5). Its “Smart Zoi” NPCs (playinzoi.com; NPC middleware context: NVIDIA ACE) run a small on-device language model for reactions plus schedule/utility autonomy underneath — i.e. even the newest entrant keeps the utility substrate and spends ML only on flavor. ⚠ details may have moved May→July 2026. |
| RimWorld / Crusader Kings III | SHIPPING | Emergent story from relationship counters + event triggers over simple state — the proof that MEMORABLE social situations need data density, not model size. |
| Generative agents (Park et al., arXiv:2304.03442, 2023; Project Sid ~1000-agent societies, arXiv:2411.00114, 2024) | RESEARCH | Memory stream + reflection + planning produces believable long-horizon social behavior; cost and latency keep it out of the per-tick loop. Our thought-ledger (mobth) + gossip (mobgo) is the same shape at integer cost. |
2. Approach & grouping: the F-formation / proxemics layer (cheap, exact, unbuilt)
Kendon's F-formation system (1990): conversational groups arrange around a shared o-space; newcomers join at recognized entry slots and the ring re-spaces. Hall's proxemics (1966) gives the distance bands: intimate <0.45 m, personal 0.45–1.2 m, social 1.2–3.6 m. HRI work has validated F-formation joining on robots for a decade (e.g. detecting and entering o-spaces). For a grid engine this is INTEGER GEOMETRY: a conversation is a ring of pose-slots around a center; approach = pathfind to the nearest open slot at the personal-distance radius; face the center; re-space on join/leave. No published system does this better with ML than with the geometry — the win is doing it at all: most games still stop NPCs at a fixed follow distance facing nowhere.
Sources: Kendon, Conducting Interaction (CUP 1990); Hall, The Hidden Dimension (1966); F-formation detection surveys in HRI (ACM/IEEE HRI proceedings, 2015–2023).
3. Conversation: what is honest at our constraints
| Approach | Status | Verdict for a sovereign, no-external-API engine |
|---|---|---|
| Cloud LLM NPC middleware (Inworld, Convai, NVIDIA ACE microservices; ACE on-device SLM demos at CES 2025) | SHIPPING (middleware) | Violates sovereignty (external calls) or GPU budget (local SLM per NPC). NOT for the per-tick loop here. |
| Ledger-driven templated dialogue: lines selected by (relationship, mood, strongest thought, gossip fact), slot-filled from the ledger | SHIPPING pattern (the entire Sims line, RimWorld barks) | BUILD THIS. Every input already EXISTS as a live door: mobfr bond, mobmood, mobth (with reasons like “soaked by rain”, “heard of your kindness”), mobgo sourced gossip. A girl who says “I heard from Mika that you saved her” out of the REAL gossip record beats a hallucination-prone model — and it is checkable by a gate. |
| Tiny on-NAS LM for paraphrase variety over the templated line | RESEARCH (ours) | Later, optional, and only as a paraphraser over ledger truth — the fact stays from the record, the wording varies. Keeps hallucination structurally impossible. |
4. Physical interaction: research vs what actually ships
| Line | Status | Finding |
|---|---|---|
| Text-to-two-person motion diffusion: InterGen + InterHuman dataset (arXiv:2304.05684, 2023); Inter-X dataset (~11k interactions, arXiv:2312.16051, CVPR 2024); reactive synthesis ReMoS (ECCV 2024); duet GPT Duolando (ICLR 2024, arXiv:2403.18811) | RESEARCH | Generation quality is good OFFLINE; runtime costs, footskate and interpenetration keep it out of shipping loops. USE AS AN AUTHORING ORACLE: generate/reference offline, bake to paired clips — never run at runtime on a no-compute web target. |
| Physics-based two-character control (RL over simulated characters, SIGGRAPH lines 2023–2025) | RESEARCH | Impressive contact realism; needs a physics sim + GPU training; wrong tier for us today. |
| Paired clips on a shared anchor: both actors authored against one alignment frame; runtime warps both roots to the anchor and plays in lockstep; IK fixups close residual contact error | SHIPPING (the industry's takedowns, hugs, carries, dances — standard practice for two decades) | BUILD THIS. It is deterministic, integer-friendly, and its correctness is GATEABLE: anchor alignment error and phase lockstep are numbers a tooth can read. This is also exactly what the coming NXA pose BANK indexes — a paired interaction = two pose tracks + one anchor + a duration. |
Do-not-build, carried over from the deformables brief and still binding here: no runtime CCD, no self-collision, no per-garment solver growth for interactions — contact plausibility comes from the paired-clip anchor + the O(1) containment layer, and the damage bound (skin strain <60%) stays the sanity check.
5. The beach engagement loop, mapped to organs that already exist
| Rung | The act | Exit criterion (falsifiable) | Instrument |
|---|---|---|---|
| S1 approach ring | Conversation = F-formation ring object; girls (and the player) claim slots at the personal-distance radius, face center, re-space on join/leave. “Woman and friends approach” = her friends (bond graph) claim adjacent slots. | Gate: every participant within [0.45,1.2] m band of ring radius, orientation error to center ≤ 15°, re-space within 60 ticks of a join; zero overlap. | new tooth (T55) — pure door reads |
| S2 engage verbs | E on a girl opens the scored interaction list (greet / chat / compliment / flirt / invite to sit / lie on towel), scored by bond + mood + advertisement, Sims-style; her side can INITIATE the same verbs at the player (P_CURI already draws her over). | Both directions occur in a soak: girl-initiated approach→engage observed without player input; verb availability tracks bond thresholds exactly. | T56 soak via doors |
| S3 ledger dialogue | Templated lines slot-filled from mobth/mobgo/mobfr — she references the REAL gossip fact and its source. | Every emitted line's fact traces to a live ledger row (gate replays the trace); zero lines without a source row. | T57 — the anti-hallucination tooth |
| S4 paired interactions | NXA pose bank + paired clips on a shared anchor: sit-together, towel lie/turn-over (pose ids 4/5 already RESERVED in the mobpose contract), hug at high bond; flirty walk-by = walk variant + target-relative gaze channel. | Anchor alignment error ≤ 1 block-q8 unit through the clip; phase lockstep exact (integer); no interpenetration beyond the containment layer's bound. | T58 + the recorder's paired channel |
Ordering note: S1–S3 are pure data + integer geometry over LIVE doors (advertisements, bond, mind ledger, mobpose) — no new render tech. S4 depends on the pose bank (the ingest-poses-as-data lane) and lands after it.
UNVERIFIED / declared gaps
- The May→July 2026 tail is unverified (search budget exhausted this session). Most likely to have moved: inZOI's Smart Zoi scope, NVIDIA ACE adoption, and the 2025–26 two-person motion papers. Re-verify + archive pass queued; until then treat every 2025+ row above as last-confirmed-May-2026.
- Citation archive pass not yet run for this brief (rule A): URLs are from the author's knowledge and must be ingested + rank-verified; any that fail get open-access mirrors and a correction ON this page.
- No public benchmark exists for interactive two-character contact at runtime — the datasets above score offline generation (FID-style), not in-engine lockstep quality; our T58 anchor-error tooth is therefore a HOUSE metric and labelled as such.
- Group conversation dynamics beyond geometry (turn-taking, interruption, who speaks next) have rich literature but no shipped-game standard to cite; S3 starts with round-robin + bond-weighted selection and declares it a heuristic.