nx_ta_parallel_grad_gate.nx
buildroot/runtime/nx_ta_parallel_grad_gate.nx
about
nx_ta_parallel_grad_gate.nx -- KEYSTONE for FAST model training: the f32 tensor autograd (ta_*) trains
DATA-PARALLEL on the sovereign thread pool. Each example in a batch runs its OWN forward+backward on a
PRIVATE (tape,vals,grads,st) arena; the per-example gradients are then reduced. The soft-float training that
bottlenecks the neural reader (and every bigger model the benchmark endgame needs) parallelizes across the
16 hardware workers here -- the conv2d pool already measured 6.76x wall-clock.
The liar-kill is STRONG: serial and parallel compute each example's gradient with identical math, so the
per-example gradients must be BIT-IDENTICAL (f32 raw equality). ANY race, arena collision, or shared-state
bug in the parallel path perturbs a bit and fails. (No f32-associativity fuzz: we compare PER-EXAMPLE grads,
not a reordered sum.)
T1 parallel per-example grads == serial, BIT-IDENTICAL across all B examples x all NP weights
T2 the pool actually ran B tasks (completed counter advanced by B) on > 1 worker
T3 deterministic (re-run parallel == identical)
expect_exit: 0 license_tier: ORIGINAL Sovereign: nx_thread_pool + nx_autograd_tensor + nx_syscalls.
dependencies 4 imports · 0 importers
imports: nx_thread_pool.nxnx_autograd_tensor.nxnx_hw.nxnx_syscalls.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| 33 | struct PgCtx |
consts
| 24 | const PG_D: i64 = 8 // input dim |
| 25 | const PG_H: i64 = 16 // hidden |
| 26 | const PG_O: i64 = 4 // output |
| 27 | const PG_NW1: i64 = 128 // H*D |
| 28 | const PG_NW2: i64 = 64 // O*H |
| 29 | const PG_NP: i64 = 192 // NW1+NW2 |
| 30 | const PG_B: i64 = 8 // batch |
| 43 | const PG_CTX_BYTES: i64 = 64 |
functions
| 20 | func pg_puts(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(1,s,n); return 0 } |
| 21 | func pg_pn(v: i64) -> i64 { let b: *u8=sys_mmap(28); var x: i64=v; if x<0{b[0]=45;sys_write(1,b,1);x=0-x} if x==0{b[0]=48;sys_write(1,b,1);return 0} var d: i64=0; var y: i64=x; while y>0{d=d+1;y=y/10} var i: i64=d-1; y=x; while i>=0{b[i]=(48+(y%10)) as u8;y=y/10;i=i-1} sys_write(1,b,d); return 0 } |
| 22 | func pg_ck(name: *u8, c: i64) -> i64 { if c==1 { pg_puts(" PASS " as *u8) } else { pg_puts(" FAIL " as *u8) } pg_puts(name); pg_puts("\n" as *u8); return c } |
| 47 | func pg_example(w: *i64, x: *i64, tgt: *i64, tape: *i64, vals: *i64, grads: *i64, st: *i64, G: *i64) -> i64 |
| 69 | func pg_task(ctx_i: i64) -> i64 calls 1: pg_example |
| 74 | func main() -> i64 |