nx_nofloat_batched_forward_gate.nx
buildroot/runtime/nx_nofloat_batched_forward_gate.nx
about
nx_nofloat_batched_forward_gate.nx -- THE DECISIVE SPECULATIVE-DECODING FOUNDATION EXPERIMENT.
Speculative decoding's speedup on MEMORY-BOUND decode = a BATCHED verify reads each weight ONCE and applies it
to K token positions, amortizing the weight-read that dominates m=1 decode. Q: does batching actually amortize
the weight-read on this box, or is it compute-bound (no gain)? This settles the whole speculative lever BEFORE
any core-engine build. Same total work (M tokens x N outputs), two loop orders on the real FFN shape:
m=1 (today's decode): M passes over the weight array (each token re-reads every weight)
m=8 (batched verify): 1 pass over the weight array (each weight read once, reused for 8 token activations)
If m=8 per-token time << m=1 -> weight-read amortizes -> speculative pays. If ~equal -> compute-bound -> it won't.
Isolated gate (i8 weights, the current fast mode); touches NO serve code. license_tier: ORIGINAL expect_exit: 0
dependencies 1 imports · 0 importers
imports: nx_syscalls.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| none |
functions
| 12 | func gp_puts(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){ n=n+1 } sys_write(1,s,n); return 0 } |
| 13 | func gp_n(v: i64) -> i64 |
| 23 | func i32x8_hsum(acc: *u8) -> i64 { let p: *i32=acc as *i32; var s: i64=0; var i: i64=0; while i<8 { s=s+(p[i] as i64); i=i+1 } return s } called by 1: main |
| 24 | func now_ms() -> i64 { let ts: *i64=sys_mmap(16) as *i64; sys_clock_gettime_mono(ts); return ts[0]*1000 + ts[1]/1000000 } |
| 25 | func lcg(st: *i64) -> i64 { st[0] = (st[0]*6364136223846793005 + 1442695040888963407) & 0x7FFFFFFFFFFFFFFF; return st[0] } called by 1: main |
| 27 | func main() -> i64 |