code wiki / (root) / nx_distributed_train_gate.nx

nx_distributed_train_gate.nx

buildroot/runtime/nx_distributed_train_gate.nx

6518 B141 linesdepth 3pulls 4 transitivereach 0 importersview sourcekind gate/proof
docsdependenciesstructsconstsfunctions

about

nx_distributed_train_gate.nx -- HEADLINE EXCEED PROOF: deterministic DISTRIBUTED training. Composes nx_fabric_collective::ring_allreduce (the byte-exact INTEGER gradient all-reduce) into a minimal no-float data-parallel trainer (forward -> backward -> all-reduce gradient -> integer optimizer step) and PROVES the trained weights are BIT-IDENTICAL regardless of the process count N (the data-parallel sharding). This is the property NO float stack has: NCCL's float all-reduce is non-deterministic across nodes, and the batch-invariant frontier (Thinking Machines) is single-node only. Ours holds because integer reduction is associative + commutative -> the summed gradient is independent of how the global batch is sharded. criteria: 1 N=1 full-batch baseline trains AND W actually MOVES (real work, not all-zeros) 2 N=2 data-parallel -> final W byte-identical to the N=1 baseline 3 N=4 data-parallel -> final W byte-identical to the N=1 baseline 4 N=8 data-parallel -> final W byte-identical to the N=1 baseline (deterministic across ALL shardings) negative control: NC1 N=8 WITHOUT the all-reduce (proc-0 local gradient only) -> W DIFFERS (the all-reduce is load-bearing) expect_exit: 0 license_tier: ORIGINAL

dependencies 2 imports · 0 importers

nx_fabric_collective.nx nx_gate_verdict.nx nx_distributed_train_gate.nx

imports: nx_fabric_collective.nxnx_gate_verdict.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main dp train dx dy ring_allreduce sys_mmap col_mod printw dp ↻ dn chk dp ↻ wmoved weq dn ↻ gv_ctr sys_mmap ↻ gv_verdict gv_puts sys_write gv_num sys_mmap ↻ sys_write ↻ sys_munmap gv_journal sys_openat_append sys_mmap ↻ gv_catn sys_mmap ↻ sys_munmap ↻ sys_now_realtime_sec sys_mmap ↻ sys_clock_gettime_real gv_cat sys_write ↻ sys_close sys_munmap ↻

structs

none

consts

none

functions

19func dp(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 }
called by 3: chkprintwmain
20func dn(v: i64) -> i64
called by 2: printwmain
30func chk(name: *u8, ok: i64) -> i64
called by 1: main calls 1: dp
34func printw(label: *u8, w: *i64, V: i64) -> i64
called by 1: main calls 2: dpdn
41func dx(s: i64, d: i64) -> i64 { return (s * 3 + d * 7 + 1) % 11 }
called by 1: train
42func dy(s: i64) -> i64 { return (s * 5 + 2) % 13 }
called by 1: train
46func train(N: i64, M: i64, V: i64, T: i64, lr_div: i64, do_ar: i64, out_w: *i64) -> i64
called by 1: main calls 3: dxdyring_allreduce
86func weq(a: *i64, b: *i64, V: i64) -> i64
called by 1: main
92func wmoved(a: *i64, V: i64) -> i64
called by 1: main
99func main() -> i64