nx_distributed_train_gate.nx
buildroot/runtime/nx_distributed_train_gate.nx
about
nx_distributed_train_gate.nx -- HEADLINE EXCEED PROOF: deterministic DISTRIBUTED training.
Composes nx_fabric_collective::ring_allreduce (the byte-exact INTEGER gradient all-reduce) into a minimal
no-float data-parallel trainer (forward -> backward -> all-reduce gradient -> integer optimizer step) and
PROVES the trained weights are BIT-IDENTICAL regardless of the process count N (the data-parallel sharding).
This is the property NO float stack has: NCCL's float all-reduce is non-deterministic across nodes, and the
batch-invariant frontier (Thinking Machines) is single-node only. Ours holds because integer reduction is
associative + commutative -> the summed gradient is independent of how the global batch is sharded.
criteria:
1 N=1 full-batch baseline trains AND W actually MOVES (real work, not all-zeros)
2 N=2 data-parallel -> final W byte-identical to the N=1 baseline
3 N=4 data-parallel -> final W byte-identical to the N=1 baseline
4 N=8 data-parallel -> final W byte-identical to the N=1 baseline (deterministic across ALL shardings)
negative control:
NC1 N=8 WITHOUT the all-reduce (proc-0 local gradient only) -> W DIFFERS (the all-reduce is load-bearing)
expect_exit: 0 license_tier: ORIGINAL
dependencies 2 imports · 0 importers
imports: nx_fabric_collective.nxnx_gate_verdict.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| none |
functions
| 19 | func dp(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 } |
| 20 | func dn(v: i64) -> i64 |
| 30 | func chk(name: *u8, ok: i64) -> i64 |
| 34 | func printw(label: *u8, w: *i64, V: i64) -> i64 |
| 41 | func dx(s: i64, d: i64) -> i64 { return (s * 3 + d * 7 + 1) % 11 } called by 1: train |
| 42 | func dy(s: i64) -> i64 { return (s * 5 + 2) % 13 } called by 1: train |
| 46 | func train(N: i64, M: i64, V: i64, T: i64, lr_div: i64, do_ar: i64, out_w: *i64) -> i64 |
| 86 | func weq(a: *i64, b: *i64, V: i64) -> i64 called by 1: main |
| 92 | func wmoved(a: *i64, V: i64) -> i64 called by 1: main |
| 99 | func main() -> i64 |