code wiki / (root) / nx_nofloat_batched_forward_gate.nx

nx_nofloat_batched_forward_gate.nx

buildroot/runtime/nx_nofloat_batched_forward_gate.nx

7926 B140 linesdepth 2pulls 2 transitivereach 0 importersview sourcekind gate/prooftopic nofloat
docsdependenciesstructsconstsfunctions

about

nx_nofloat_batched_forward_gate.nx -- THE DECISIVE SPECULATIVE-DECODING FOUNDATION EXPERIMENT. Speculative decoding's speedup on MEMORY-BOUND decode = a BATCHED verify reads each weight ONCE and applies it to K token positions, amortizing the weight-read that dominates m=1 decode. Q: does batching actually amortize the weight-read on this box, or is it compute-bound (no gain)? This settles the whole speculative lever BEFORE any core-engine build. Same total work (M tokens x N outputs), two loop orders on the real FFN shape: m=1 (today's decode): M passes over the weight array (each token re-reads every weight) m=8 (batched verify): 1 pass over the weight array (each weight read once, reused for 8 token activations) If m=8 per-token time << m=1 -> weight-read amortizes -> speculative pays. If ~equal -> compute-bound -> it won't. Isolated gate (i8 weights, the current fast mode); touches NO serve code. license_tier: ORIGINAL expect_exit: 0

dependencies 1 imports · 0 importers

nx_syscalls.nx nx_nofloat_batched_forward_gat

imports: nx_syscalls.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap gp_puts sys_write lcg now_ms sys_mmap ↻ sys_clock_gettime_mono i32x8_hsum gp_n sys_write ↻ sys_mmap ↻

structs

none

consts

none

functions

12func gp_puts(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){ n=n+1 } sys_write(1,s,n); return 0 }
called by 1: main calls 1: sys_write
13func gp_n(v: i64) -> i64
called by 1: main calls 2: sys_writesys_mmap
23func i32x8_hsum(acc: *u8) -> i64 { let p: *i32=acc as *i32; var s: i64=0; var i: i64=0; while i<8 { s=s+(p[i] as i64); i=i+1 } return s }
called by 1: main
24func now_ms() -> i64 { let ts: *i64=sys_mmap(16) as *i64; sys_clock_gettime_mono(ts); return ts[0]*1000 + ts[1]/1000000 }
called by 1: main calls 2: sys_mmapsys_clock_gettime_mono
25func lcg(st: *i64) -> i64 { st[0] = (st[0]*6364136223846793005 + 1442695040888963407) & 0x7FFFFFFFFFFFFFFF; return st[0] }
called by 1: main
27func main() -> i64