Nishi Audio — Sovereign Audio Stack

Capability & state-of-the-art status. Every organ is sovereign: authored in NishiLang, compiled nx_cc → nxasm_x86 — no gcc, no borrowed decoders, from the first byte up.

Measured 2026-07-21 by building & running the real gates across the ~70-organ audio surface. Green = a gate passed with real numbers, verified independently. Nothing on this page is asserted without a gate.

Nishi audio is not greenfield and it is not yet SOTA. It is a large, proven, but scattered surface being consolidated onto an object-oriented base while it climbs toward the neural state of the art. This page is the honest scorecard.

Proven capabilities (gate-verified)

Capability	Organ / gate	Verdict	Measured evidence
Lossless audio codec	`nx_audio_lossless_gate`	GREEN	Bit-exact (max-err = 0) on all 3 paths; FLAC-class LPC; 96 kHz wide-spectrum retained
Voice codec (NishiVoice v2)	`nx_voice_codec_v2`	GREEN 3/3	Real LPC autocorr→Levinson→synth→range-code; 2.4× compression, faithful reconstruction
Match a sampled voice (identity)	`nx_voiceprint`	GREEN	LPC-PARCOR vocal-tract embedding; same/diff-voice separation 92×; tamper rejected
Clone a captured voice	`nx_voice_clone_synth`	GREEN	Re-synth in the target tract; clone→target 48k vs clone→other 1.65B (33900× margin)
Text → phonemes	`nx_g2p`	GREEN	Rule-based LTS incl. digraphs (sh/th/ch/ee/oo/ck); 27-phoneme inventory
Packet-loss concealment	`nx_audio_dred_gate`	GREEN 4/4	Burst of 8 → 0 gaps recovered vs 7 gaps for 1-frame FEC
Audiobook ingest (MP3/ID3 → .nmz)	`nx_audio_gate`	GREEN 10/0	ID3 parse + byte-exact carry + CID; RED→GREEN debt eaten 2026-07-21 (recursive mkdir fix)

New: read a book in different generated voices → WAV

nx_book_voice_read — the end-to-end assembler built 2026-07-21, composing only proven-green parts (nx_g2p → nx_voice_read per-speaker → concat → nx_wav header), routing around a broken legacy runner. GREEN 4/4, run-exit 0.

Writes a real RIFF / WAVE, Microsoft PCM, 16-bit, mono 8 kHz file — 116 444 bytes verified on disk.
Narrator vs character are measurably distinct voices: distance ratio 77× the same-voice baseline (warp 1.00/F0 120 vs warp 1.20/F0 175).
g2p("ship") → [SH, IH, P]; SH renders as hiss (0 voiced frames), IH as tone (5 voiced frames) — text became correct speech.

Honest SOTA status

nx_voicegen_census (GREEN, adversarial neg-control rejected) grades the voice stack against neural SOTA: BEHIND = 10 axes, PARITY = 0, AHEAD = 2 (both on the sovereignty floor). Formant source-filter synthesis is intelligible but caps at roughly MOS 3 — robotic. This is stated plainly, not hidden.

The SOTA path is neural, on the existing f32/SSE substrate: a neural vocoder (HiFi-GAN-class), a neural acoustic model (VITS / FastSpeech2-class), and neural voice cloning (EnCodec / VALL-E-class), with nx_voice_eval as the MOS-proxy ruler. That is the lane's north star, filed as tracked work — not claimed as done.

Architecture — object-oriented consolidation

A real codec base class already exists: nx_codec (vtable: encode / decode / frame_cas_hash / grade_card; sealed enums). The consolidation, without losing any functionality:

Adopt nx_codec for v2 + lossless (only the v1 adapter uses it today).
De-duplicate the lossless LPC codec (two near-identical copies, diff-proven) into one shared module; add partitioned-Rice + adaptive order to close sub-FLAC efficiency.
Extract a shared nx_formant_core and nx_wav_writer base (formant/RIFF logic is copy-pasted 4–6×).
Unify the two LPC lineages once the nx_syscalls vs nx_syscalls_x86_64 two-tree split is resolved (the root cause of most audio duplication).

Ownership & tracked work

RACI R = engineer-audio · A = pm · C = librarian · I = team — registered on the sovereign common-tasks plane 2026-07-21 (audio was previously absent from the RACI SSOT).

Debt Five measured debts filed (seq 498–502): legacy TTS hang, formant copy-paste, neural-SOTA gap, lossless-codec duplication, tree-wide mkdir bug (partially eaten). Build order book-voice pipeline (done) → publish → codec consolidation → neural vocoder rung.