Nishi Family › Compare › GPU Worker Mesh and Distributed Inference
Nishi vs the Field
GPU Worker Mesh and Distributed Inference — mechanically measured, liar-killed, sovereign.
A NAS orchestrator dispatching capability-scoped jobs to GPU workers (laptop 5080 images, west 3090 video, cloud) and pushing results to the gallery -- Nishi vs NVIDIA Triton and Ray Serve and Modal and KServe
How this is scored. Every Nishi cell is measured: the generator reads the real organ source on disk and requires the implementing symbol to exist (no self-grading). Competitor cells record documented capability presence. Best=leads this axis, Yes=present, Part=partial, No=absent. This is capability presence, not depth or scale: the majors lead on index size and neural ranking. Nishi's genuine exceeds are the sovereignty / neutrality / determinism axes.
Feature matrix
| Capability | Nishi | NVIDIA Triton | Ray Serve | Modal | KServe | Notes |
|---|---|---|---|---|---|---|
| Capability-scoped job dispatch | Yes | Yes | Best | Best | Yes | Ray and Modal are Best at programmatic dispatch; Nishi routes by job type to the right worker class and enforces the grant deny-by-default |
| Cross-host GPU worker routing | Yes | Yes | Best | Best | Best | Ray and K8s-native KServe route across a cluster as Best; Nishi routes image to the 5080 and video to the 3090 and other to cloud |
| Sovereign end-to-end actuation | Yes | Yes | Yes | Yes | Yes | All frameworks serve inference; Nishi proved a single organ POST-decode-publish loop over its own TCP with zero shell |
| Native GPU container governance daemonless | Yes | Part | Part | Part | Part | Triton and Ray need a server daemon and Modal is cloud and KServe needs Kubernetes; Nishi governs the worker natively via Job Objects no daemon no VM |
| Content-addressed model and artifact registry | Yes | Best | Yes | Best | Best | Triton model repository is the reference; Nishi has a SHA-256 content-addressed registry with integrity-verified pull |
| Push results to consumable storage | Yes | Yes | Yes | Yes | Yes | All can write outputs; Nishi decodes the worker response and publishes to the gallery in the same sovereign organ |
| Model load and unload | Yes | Best | Yes | Best | Best | Triton and Modal do dynamic model load and unload as Best; Nishi loads and unloads the worker (VRAM 0 to 10.9GB to 0) but manually not yet on demand |
| Worker health probe and liveness | Yes | Best | Best | Best | Best | Mature frameworks have rich health and failover as Best; Nishi has a basic reachability probe |
| Dynamic request batching | Yes | Best | Yes | Yes | Yes | Triton fuses requests into one GPU forward pass; Nishi has a sovereign batch SCHEDULER (coalesce by preferred-size OR max-delay, sim 32 into 4 batches of 8) AND live n=K batched EXECUTION (proven: genbatch 3 = 3 images in ONE round-trip on the 5080); MEASURED-HONEST -- the current sd.cpp engine runs n greater than 1 SEQUENTIALLY (20.3s per image batched vs 19.3s sequential) so today the win is fewer dispatches and connections not fewer forward passes; true fused throughput needs a batching-capable backend -- Nishi now HAS the compute primitive (nx_batch_infer, a sovereign fused batched-GEMM, bit-exact vs sequential via weight-stationary reuse); the CPU reuse win is modest (compute-bound), the large win is GPU kernel-launch amortization |
| Multi-worker load balancing across replicas | Yes | Best | Best | Best | Best | Balancing across replicas is Best in every framework; Nishi now health-probes each replica (bounded 1s non-blocking) and round-robins among the healthy ones skipping the down -- PROVEN live routing to the laptop 5080 and skipping the offline west 3090; west or a cloud replica joins as one endpoint row |
| Autoscaling scale-to-zero on demand | Yes | Yes | Best | Best | Best | Modal and KServe scale-to-zero are Best; Nishi has a proven policy controller AND now a HANDS-OFF host-agent (nx_mesh_hostagent) that autonomously forks the worker on demand and reaps it on idle -- PROVEN with a real process (probe 0 to 1 to 0, native fork and kill lifecycle); Nishi idle is a TRUE 0 MB 0 procs so scale-to-zero costs almost nothing |
| Streaming responses | Yes | Yes | Best | Best | Yes | Token and chunk streaming is standard; Nishi now emits a valid Server-Sent-Events stream -- accepted then one progress event per step (step total pct) then a terminal result with the artifact URL, consumable by a browser EventSource and the same shape as our live /api/events; the natural frame-delivery channel for video on the west 3090 |
| Request queue and admission control | Yes | Best | Best | Best | Yes | Queueing and backpressure protect the GPU under load; Nishi now has a bounded queue with admission control -- sim proved 100 requests at cap 64 admits 64 and sheds 36 with backpressure, never over-cap, and no request starves |
| Multi-model multiplexing on one GPU | Yes | Best | Best | Best | Best | Serving many models on shared GPU memory is Best in the frameworks; Nishi now has a VRAM-budgeted LRU model residency -- route by model-id, keep the hot models loaded, evict the least-recently-used to fit a new one, refuse an oversized model; composes with autoscale load-unload and the content-addressed model registry |
| EXCEED sovereign: own-stack no serving framework | Best | No | No | No | No | The whole mesh is our own TCP client plus base64 plus container governance plus dispatch -- no Triton no Ray no CUDA serving runtime no cloud dependency |
| EXCEED unique: capability-scoped resource access | Best | No | No | No | No | A job may only call the worker and storage its capability grants -- deny-by-default MCP-style trusted-to-do-work enforcement none of the frameworks have this model |
| EXCEED sovereign: zero-cost on your hardware auditable | Best | No | No | No | No | Runs on your own GPUs at no per-call cost with the whole loop readable bits-up; the managed frameworks are cloud-metered or heavyweight infra |
coverage 1000/1000EXCEEDS 3PRESENT 14ABSENT 0
Honest verdict. Nishi has broad, sovereign search-engine presence — a real organ behind every core ranking and crawl axis, built bits-up in one language. It is not yet a scale or neural-ranking rival: the majors lead on index size (billions to trillions vs a Common-Crawl-seeded slice) and trained rerankers. Nishi's genuine exceeds are structural — owning the whole stack, neutrality by construction, and a zero-JS integer-deterministic result page. The climb is a bigger index and a trained reranker.
Generated by nx_swcompare_matrix (sovereign NishiLang organ) from knowledge/compare/workermesh.matrix — every Nishi cell verified against real organ source on disk. Zero JS, zero trackers.