Nishi Family › Compare › Sovereign Model Lifecycle (Modelwright)
Nishi Compare · measured, not asserted
Sovereign Model Lifecycle (Modelwright)
Nishi vs the field — every Nishi cell is measured against real organ source at emit time; each gap names the watch contract that will close it.
Nishi vs PyTorch and HuggingFace and llama.cpp and Megatron-LM
Layer 1 · Executive
Where we are. Measured 2026-08-19. The loop is REAL and toy-scale: autodiff, an integer no-float transformer that greedy-matches the float reference, BPE, K-quants, GGUF load of Qwen2.5-0.5B and an OpenAI-shaped serve. The matrix was last scored 2026-07-10 and the nofloat lane has since shipped a MoE forward, Muon and AdamW optimisers, distillation, linear and latent attention, MXFP4 and MXFP8 and a DDP autograd gate -- substrate the rungs below compose rather than re-create. What is still absent is alignment (SFT, DPO, LoRA), a standard eval harness, fused kernels, parallel training, GPU training and a model at real scale.
Where we need to go. Train and align a real model on our own substrate: alignment first (the cheapest path to a usable seat), then a harness that cannot be gamed, then kernels, parallelism and silicon, then a 100M-plus pretrain -- every step bit-exact reproducible, the structural exceed the float field cannot match.
The unit. 1 u = one measured session-leg. Calibration from landed rungs: the mangagen panel compositor went from existing substrate to shipped and live-verified in ONE leg (2026-08-13); the citations rung went from 3 to 55 domains in one leg across seven seats (2026-08-18); a greenfield engine with a bite-proven gate has measured 2 to 4 legs. Estimates recalibrate as rungs land and PR7 actuals write back.
Cost to an aligned seat: 5.5 u. Through M0. SFT, LoRA and DPO on the existing integer LM; quality is measured by held-out rules declared before the runs.
Cost to scale architecture: 16 u. Through M2: the harness, fused attention, MoE training, parallel training and a hub.
Cost to a real model: 26 u. Everything below. Pretraining at scale is the deepest rung in the ledger and depends on every kernel and parallel rung before it.
10 of 21 capabilities measured|2 of them measured exceeds|11 open|coverage 476/1000|adoption 6 full / 4 partial
Layer 2 · Roadmap
Do this next — computed by the ranker, never chosen by a seat
Order from nx_compare_rank (nx_dr_ocm: (deficit + cost-of-delay + option + enables) x sponsor x self-sufficiency x momentum / cost). FINISH rows are rungs whose symbol is present but whose organ is short of full adoption: the cheapest closures on this board, listed before any new work. Stamp: # asof=1787883516 domain=modelwright target_version=0.1 rungs=11 done=0 open=11 finish=0 ranker=nx_dr_ocm
| # | Stage | Rung | Priority | Derivation |
|---|---|---|---|---|
| #1 | 0.1 | LoRA adapters (R1) lora_apply | 466 | v=7 m=1 c=15 |
| #2 | 0.1 | Instruction SFT pipeline (R0) sft_run | 400 | v=8 m=1 c=20 |
| #3 | 0.1 | DPO preference alignment (R2) dpo_step | 300 | v=6 m=1 c=20 |
| #4 | later | Speed published vs llama.cpp (R8) ts_bench | 1200 | v=12 m=1 c=10 |
| #5 | later | Flash-attention class fused kernel (R4) fa_fused | 750 | v=15 m=1 c=20 |
| #6 | later | Model hub (R10) mh_publish | 500 | v=5 m=1 c=10 |
| #7 | later | Standardized eval harness (R3) eh_run | 300 | v=6 m=1 c=20 |
| #8 | later | Mixture-of-experts training (R5) moe_train_step | 280 | v=7 m=1 c=25 |
| #9 | later | GPU training step (R7) gpu_train_step | 275 | v=11 m=1 c=40 |
| #10 | later | Pretraining at real scale (R9) ps_run | 200 | v=10 m=1 c=50 |
| #11 | later | Tensor and data-parallel training (R6) tp_shard | 133 | v=4 m=1 c=30 |
Critical path — contract, done-rule, executor, cost
| Rung | Closes with | Definition of done (pre-declared) | Executor | Est. |
|---|---|---|---|---|
| Instruction SFT pipeline (R0) | sft_run | Supervised fine-tuning over the 100 percent integer LM on a banked instruction set; held-out exact-match beats the base model by a PRE-DECLARED margin; the run is bit-exact reproducible on a second CPU (the exceed this lane already holds extends to training) | Organ | 2 u |
| LoRA adapters (R1) after R0 | lora_apply | Rank-r adapters on the attention projections trained integer; merge reproduces the full-weight result bit-exact; the existing adapter pool registry (nx_lora_pool) is the store, not a new one | Organ | 1.5 u |
| DPO preference alignment (R2) after R0 | dpo_step | DPO loss over banked preference pairs; win-rate against the SFT model on held-out pairs PRE-DECLARED; a run that does not beat SFT stays unwired | Organ | 2 u |
| Standardized eval harness (R3) | eh_run | MMLU and HumanEval-class tasks run from banked datasets with exact-match and pass-at-1; the harness is shared with the referee domain (its std_bench row names the same capability) so one ruler grades every model | Organ | 2 u |
| Flash-attention class fused kernel (R4) | fa_fused | Tiled attention that never materialises the score matrix; bit-identical to nx_attention at every tested length; measured speedup over the plain path at 2k context is published as a number | Organ | 2 u |
| Mixture-of-experts training (R5) after R4 | moe_train_step | Router plus experts trained end to end in integer (the nofloat lane already ships the MoE forward and its arch gate); load-balance loss; held-out beats dense at equal active parameters, PRE-DECLARED | Organ | 2.5 u |
| Tensor and data-parallel training (R6) after R4 | tp_shard | Sharded GEMM across processes over the mesh transport; bit-exact against the single-process run (the nofloat DDP autograd gate is the precedent); scaling efficiency printed per worker count | Organ | 3 u |
| GPU training step (R7) after R4 | gpu_train_step | The integer train step executes on the 5080 through the sovereign GPU lane; step bits equal the CPU step; tokens per second published | Organ | 4 u |
| Speed published vs llama.cpp (R8) after R4,R7 | ts_bench | Decode and train tokens per second measured on the same box as llama.cpp with the same model; the ratio is a ratchet floor that refuses regression | Organ | 1 u |
| Model hub (R10) | mh_publish | Content-addressed model cards plus weights served from the estate with sha-verified pull; composes the container registry CAS already proven | Organ | 1 u |
| Pretraining at real scale (R9) after R5,R6,R7 | ps_run | A 100M-plus parameter integer model pretrained on the banked corpus to a PRE-DECLARED held-out perplexity; checkpoints content-addressed; the run is resumable and bit-exact | Local model | 5 u |
Milestones
| Milestone | Rungs | Cumulative |
|---|---|---|
| M0 · Align what we have | R0,R1,R2 | 5.5 u |
| M1 · Kernels and an honest harness | R3,R4 | 9.5 u |
| M2 · Scale architecture | R5,R6,R10 | 16 u |
| M3 · Silicon | R7,R8 | 21 u |
| M4 · A real model | R9 | 26 u |
Layer 3 · Engineering
How this is scored. Every Nishi mark is measured: the generator reads the real organ source on disk and requires the implementing symbol to exist (no self-grading). A watching tag names the organ and symbol contracted to close a gap — the mark flips itself on the next compare beat when that workstream ships, and the
comparewatch- plane row flips with it. The flip is necessary, not sufficient: it proves the symbol exists, never that the capability is good. The bar is the rung's pre-declared done-rule, proven by its gate — a symbol shipped without the behaviour behind it is a defect, and the flip is exactly what makes that defect visible instead of quiet. Competitor marks record documented capability presence — presence, not depth or scale. Adoption is measured too: every measured row carries where its organ stands on the estate's ladder (source → built → promoted → registered → invoked; libraries by importer reach minus validation importers; gates by the execution surfaces that run them). A row is fully adopted only at the top of its ladder; anything short is tagged partial with the exact remedy, so a build nobody promoted can no longer read as shipped. Census stamps: importers asof 1787849099, gate census asof 1787855507 (unix seconds; -1 = census absent).Capability matrix — measured against source
◉ leads / measured exceed● present◐ partial○ absent · click any capability for its evidence
| Capability | Nishi | PyTorch | HuggingFace | llama.cpp | Megatron-LM |
|---|---|---|---|---|---|
Reverse-mode autodiff training substrateMeasured:ta_new exists in runtime/nx_autograd_tensor.nx, verified at emit. Tensor reverse-mode autograd; the first sovereign LM trained from scratch beat the analytic midpoint bit-exact (role scorecard MODELWRIGHT, 2026-06-10); PyTorch autograd is the bar [pytorch-autograd] Adoption: SOURCE-ONLY — PARTIAL: source exists, never compiled: /api/build it. | ● | ◉ | ● | ○ | ● |
Transformer forward (100 percent integer no-float)Measured:qmul exists in runtime/nx_nofloat_llm.nx, verified at emit. The core forward every model has, done with zero floating point Adoption: LIB-WIRED importers=30 nonval=7 — fully adopted (top of its ladder). | ● | ● | ● | ● | ● |
BPE tokenizer trainingMeasured:nx_bpe_vocab_new exists in runtime/nx_bpe.nx, verified at emit. Own byte-pair tokenizer + vocab training Adoption: SOURCE-ONLY — PARTIAL: source exists, never compiled: /api/build it. | ● | ● | ● | ● | ● |
Integer K-quant weightsMeasured:nx_q4k_dot_row_col exists in runtime/nx_q4k_matmul.nx, verified at emit. Q4-K dot kernels; llama.cpp K-quants are the CPU bar [llamacpp]; TRT/Megatron are GPU-float Adoption: LIB-WIRED importers=14 nonval=9 — fully adopted (top of its ladder). | ● | ● | ● | ◉ | ○ |
SIMD integer GEMMMeasured:i32x8_hsum exists in runtime/nx_nofloat_gemm.nx, verified at emit. AVX2 integer GEMM (7x pooled); the field's GEMM is float/cuBLAS Adoption: LIB-GATE-ONLY importers=3 — PARTIAL: imported only by validation organs (gates, tests, benches): wire it into a shipping program. | ● | ◉ | ● | ◉ | ● |
Open-weights GGUF loadingMeasured:nx_gguf_load_model_weights exists in runtime/nx_gguf_load_model.nx, verified at emit. Real Qwen2.5-0.5B loads and greedy-matches the float reference 4-for-4 (2026-07-09); llama.cpp is the GGUF home Adoption: LIB-WIRED importers=2 nonval=1 — fully adopted (top of its ladder). | ● | ◉ | ◉ | ◉ | ● |
Served chat API (OpenAI-shaped)Measured:ls_extract_prompt exists in runtime/nx_f32_llm_serve.nx, verified at emit. Sovereign seat on :11434 serving real Qwen; vLLM/llama.cpp lead production serving Adoption: RUN-BY:daemon — fully adopted (top of its ladder). | ● | ● | ◉ | ◉ | ○ |
Attention and layernorm forwardMeasured:nx_attn_score_matrix exists in runtime/nx_attention.nx, verified at emit. Score-matrix attention + softmax; the transformer core Adoption: LIB-WIRED importers=8 nonval=5 — fully adopted (top of its ladder). | ● | ● | ● | ● | ● |
Large-scale pretraining (billions of params, trillions of tokens)Open — watchingruntime/nx_pretrain_scale.nx : ps_run, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. Megatron/PyTorch FSDP pretrain frontier models; ours is a toy LM | ○ | ◉ | ● | ● | ◐ |
GPU / CUDA trainingOpen — watchingruntime/nx_gpu_train.nx : gpu_train_step, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. CUDA/ROCm training is the bar; Nishi trains CPU-integer by design today | ○ | ◉ | ○ | ● | ◐ |
Distributed tensor-parallel trainingOpen — watchingruntime/nx_tensor_parallel.nx : tp_shard, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. Megatron-LM 3D parallelism is the bar [megatron-lm]; Nishi single-process | ○ | ● | ○ | ○ | ◉ |
Instruction fine-tuning (SFT) pipelineOpen — watchingruntime/nx_sft.nx : sft_run, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. HuggingFace TRL SFTTrainer is the bar [trl]; no SFT loop here | ○ | ● | ◐ | ○ | ● |
RLHF / DPO preference alignmentOpen — watchingruntime/nx_dpo.nx : dpo_step, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. TRL DPO/PPO alignment is the bar [trl] [dpo2023]; the models run un-aligned | ○ | ● | ◐ | ○ | ● |
LoRA / parameter-efficient fine-tuningOpen — watchingruntime/nx_lora.nx : lora_apply, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. PEFT/LoRA adapters are the bar [lora2021]; absent here | ○ | ● | ◐ | ● | ● |
Standardized eval harness (MMLU / HumanEval)Open — watchingruntime/nx_eval_harness.nx : eh_run, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. lm-eval-harness is the field standard [lm-eval-harness]; ours is qabench (referee census) not a model-eval harness | ○ | ● | ◉ | ● | ● |
Flash-attention / fused training kernelsOpen — watchingruntime/nx_flash_attn.nx : fa_fused, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. FlashAttention fused kernels are the bar [flashattn2022]; ours is a plain score matrix | ○ | ◉ | ● | ● | ◐ |
Mixture-of-experts trainingOpen — watchingruntime/nx_moe_train.nx : moe_train_step, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. Sparse MoE training (Megatron/DeepSpeed); Nishi dense-only | ○ | ● | ● | ● | ◉ |
Competitive train and inference speedOpen — watchingruntime/nx_train_speed.nx : ts_bench, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. Measured 2.4 tok/s CPU decode = 8-25x behind llama.cpp-class; GPU stacks train orders faster | ○ | ◉ | ● | ◉ | ◉ |
Model hub and distributionOpen — watchingruntime/nx_model_hub.nx : mh_publish, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. HuggingFace Hub is the bar [hf-transformers]; our models live on the NAS, not a public hub | ○ | ● | ◉ | ● | ○ |
100 percent integer no-float train and inference (bit-exact reproducible by construction)Measured exceed:qmul in runtime/nx_nofloat_llm.nx, verified at emit. A 100 percent integer LLM greedy-matches the float reference 4-for-4 and reproduces bit-identically on any CPU; the entire field is float and nondeterministic across runs/hardware Adoption: LIB-WIRED importers=30 nonval=7 — fully adopted (top of its ladder). | ◉ | ○ | ○ | ○ | ○ |
Sovereign whole training-and-serving loop (own language, compiler, kernels; zero PyTorch/CUDA/BLAS)Measured exceed:ta_new in runtime/nx_autograd_tensor.nx, verified at emit. Train + quantize + serve entirely on our own stack bits-up; PyTorch/Megatron sit on CUDA + cuBLAS, llama.cpp on foreign C++ toolchain -- our brain runs on OUR substrate, the literal own-the-loop Adoption: SOURCE-ONLY — PARTIAL: source exists, never compiled: /api/build it. | ◉ | ○ | ○ | ○ | ○ |
On these two registers. Rows are declared in the domain's plan file and carry the debt id, which is the join key back to the sovereign debt plane — that plane, not this page, is the authority on state. Reconciling them automatically (the regen reading the plane and refreshing these rows) is a named, owed rung; until it lands, treat an id here as a pointer to look up, not a status to trust.
Honest verdict. The Modelwright owns Nishi's own brain -- and the honest headline is that it is REAL but TOY-scale. It trains from scratch (a reverse-mode autodiff substrate; the first sovereign LM beat the analytic midpoint bit-exact, 2026-06-10), it runs a 100 percent integer no-float transformer that greedy-matches the float reference 4-for-4 (2026-07-09), it loads real open weights (Qwen2.5-0.5B via GGUF), tokenizes with its own BPE, quantizes to integer K-quants, and serves an OpenAI-shaped chat API -- the whole loop is ours, our brain not Claude's. But at production scale it is not close: no large-scale pretraining, no GPU or CUDA training, no distributed tensor-parallelism (Megatron), no instruction fine-tuning or RLHF/DPO alignment (TRL), no LoRA, no standardized eval harness, and measured CPU decode is 8-25x behind llama.cpp-class. The two genuine exceeds are structural, not scale: 100 percent integer training and inference is bit-exact reproducible BY CONSTRUCTION where the entire field is float and nondeterministic, and the whole stack is ours bits-up -- own language, compiler, kernels, zero PyTorch/CUDA/BLAS beneath. The climb is the deepest rung in the ledger: train a real model at real scale on our own substrate, then alignment and an eval harness (which the referee shares).
Person · product · place — not yet measured for this domain
Every compare carries this layer. Declare
knowledge/compare/modelwright.ppp (rows surface|nishi or c1..c4|label|url|connect naming OUR live surface and each rival's front door), run nx_ppp_probe domain modelwright, and this section fills itself on the next beat: the same ruler on both sides — privacy and CX (third-party hosts, tracker classes, cookies, security headers), design and longevity (design hygiene, computed WCAG contrast, render-blocking resources, unsized media, script weight, theme and motion queries), findability (landmarks, skip link, on-site search, breadcrumb, headings, internal links).References
Beyond a link list. Every reference below resolves twice — the publisher's copy and, where banked, the estate's own non-rottable library mirror with a content pin — and carries its evidence class plus the exact claim on this page it grounds. Keyed marks like [key] in the matrix notes jump here. A dash means honestly absent, never assumed.
- [pytorch-autograd] PyTorch. Automatic differentiation package -- torch.autograd. PyTorch 2.13 documentation. Accessed August 2026. publisher · read in our library
knowledge/fetched/cmp_modelwright_pytorch-autograd.html· pinh6a11d941924bc30f6b0bda0181b07a7df63c84650e9bd97e7efd0b8408c14c6f· accessed 2026-08-18 · vendor-docGrounds: The PyTorch column and the BARS line (PyTorch v2.13.0): the Reverse-mode autodiff training substrate row (PyTorch autograd is the bar nx_autograd_tensor is graded against) and the GPU / CUDA training and Large-scale pretraining cells where PyTorch reads Best. - [hf-transformers] Hugging Face. Transformers documentation (state-of-the-art pretrained models for inference and training; the Hub-backed model library). Accessed August 2026. publisher · read in our library
knowledge/fetched/cmp_modelwright_hf-transformers.html· pinh6047febd65c25a6bd5744107f0f82c0b62517dd6e709469431bdb7f1d431b8d5· accessed 2026-08-18 · vendor-docGrounds: The HuggingFace column and the BARS line (Transformers v5.13.0): Model hub and distribution Best (HuggingFace Hub is the bar; our models live on the NAS), Open-weights GGUF loading and Served chat API cells, and the Standardized eval harness column code. - [trl] Hugging Face. TRL -- Transformer Reinforcement Learning: SFTTrainer, DPOTrainer, PPO/GRPO and reward-model training documentation. Accessed August 2026. publisher · read in our library
knowledge/fetched/cmp_modelwright_trl.html· pinhdd3f645f67ca35d24e6f7dd736e2000cb9377df1a052a59c362553e494e519c8· accessed 2026-08-18 · vendor-docGrounds: The BARS line (TRL v1.8.0) and two absent rows: Instruction fine-tuning (SFT) pipeline (HuggingFace TRL SFTTrainer is the bar; no SFT loop here) and RLHF / DPO preference alignment (TRL DPO/PPO alignment is the bar; the models run un-aligned). - [llamacpp] ggml-org. llama.cpp: LLM inference in C/C++ (project README, GitHub). Accessed August 2026. publisher · read in our library
knowledge/fetched/cmp_llm_llamacpp.html· pinh8f4ccd064eee25e9c5294a49af67e94126cf3c7c21070677fc7c0eb55eb8a1a0· accessed 2026-08-18 · vendor-docGrounds: The llama.cpp column and BARS line: Integer K-quant weights Best (llama.cpp K-quants are the CPU bar), Open-weights GGUF loading (llama.cpp is the GGUF home), SIMD integer GEMM Best, and the Competitive train and inference speed row (measured CPU decode 8-25x behind llama.cpp-class). - [megatron-lm] NVIDIA. Megatron-LM and Megatron Core: GPU-optimized library for training transformer models at scale -- tensor, pipeline, sequence, context and expert parallelism (project README, GitHub). Accessed August 2026. publisher · read in our library
knowledge/fetched/cmp_modelwright_megatron.html· pinh20fd935ac0734062eec78ab73cfcfbcb93946803031f531a893f73e68aa4a163· accessed 2026-08-18 · vendor-docGrounds: The Megatron-LM column and BARS line: Distributed tensor-parallel training (Megatron-LM 3D parallelism is the bar; Nishi single-process), Large-scale pretraining (Megatron/PyTorch FSDP pretrain frontier models) and Mixture-of-experts training (sparse MoE training, Megatron/DeepSpeed). - [lora2021] Hu, Shen, Wallis, Allen-Zhu, Li, Wang, Wang, Chen. LoRA: Low-Rank Adaptation of Large Language Models. ICLR 2022. arXiv:2106.09685. publisher · read in our library
knowledge/fetched/cmp_modelwright_lora2021.html· pinh70bb70b4fd6d41b8a0f53fc5148017bf6ef5b73acba822027397f861aee135c6· accessed 2026-08-18 · published-paperGrounds: The LoRA / parameter-efficient fine-tuning row (PEFT/LoRA adapters are the bar; absent here) and the verdict line naming no LoRA as a scale gap: this paper defines the low-rank adapter the nx_lora contract would implement. - [dpo2023] Rafailov, Sharma, Mitchell, Ermon, Manning, Finn. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. NeurIPS 2023. arXiv:2305.18290. publisher · read in our library
knowledge/fetched/cmp_modelwright_dpo2023.html· pinhc213a47bb5c67ad1983741bec43fc5e6ab252ae9737bd3e6c8daa2edbf5a3fe3· accessed 2026-08-18 · published-paperGrounds: The RLHF / DPO preference alignment row: DPO is the reward-model-free preference objective TRL ships as DPOTrainer, i.e. the alignment bar the nx_dpo contract is named after; the models run un-aligned today. - [flashattn2022] Dao, Fu, Ermon, Rudra, Re. FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness. NeurIPS 2022. arXiv:2205.14135. publisher · read in our library
knowledge/fetched/cmp_modelwright_flashattn2022.html· pinh070112e924e64cb514df26ecf174263f5d3343560438a88086d4a7c478547f64· accessed 2026-08-18 · published-paperGrounds: The Flash-attention / fused training kernels row (FlashAttention fused kernels are the bar; ours is a plain score matrix) and the Attention and layernorm forward row it is measured beside -- IO-aware tiling is what nx_attention lacks. - [lm-eval-harness] EleutherAI. Language Model Evaluation Harness: a unified framework for few-shot evaluation of language models (MMLU, HumanEval-class tasks, 60+ benchmarks; project README, GitHub). Accessed August 2026. publisher · read in our library
knowledge/fetched/cmp_modelwright_lm-eval.html· pinhb67a0fef3d08e781f2a2093997dde3f809f793714a5638dc0355a6823dfd4455· accessed 2026-08-18 · vendor-docGrounds: The Standardized eval harness (MMLU / HumanEval) row: lm-eval-harness is the field standard the nx_eval_harness contract is measured against; ours is qabench (referee census), not a model-eval harness.
generated by nx_swcompare_matrix (sovereign NishiLang organ) from knowledge/compare/modelwright.matrix · every Nishi cell verified against organ source at emit time · watch cells re-measured on every compare beat · zero JS, zero trackers