nx_mesh_batch.nx
buildroot/runtime/nx_mesh_batch.nx
about
nx_mesh_batch.nx -- WORKER MESH: DYNAMIC BATCHING scheduler + request QUEUE with admission control (the Triton
dynamic-batcher analog). A GPU serves far more throughput if concurrent requests are COALESCED into one forward
pass instead of run one-at-a-time. This organ is the SCHEDULER that sits in front of a worker:
* QUEUE (bounded FIFO) + ADMISSION CONTROL: admit while qlen < cap, else REJECT (backpressure / 503) -- protects
the GPU from unbounded pileup under load.
* DYNAMIC BATCH COALESCER: form + dispatch a batch when EITHER the queue reaches the preferred batch size
(qlen >= max_batch) OR the oldest queued request has waited >= max_delay_ms (bound tail latency) -- exactly
Triton's {preferred_batch_size, max_queue_delay}. A dispatched batch of K maps to an n=K worker call.
The scheduler is pure integer policy -> mechanically provable. A discrete-event SIM feeds an arrival stream and
reports batches formed, avg batch size (the throughput win), rejects, and max queue depth.
CLI: (no args) -> self-test GATE (policy table + sim + neg-controls)
sim <burst> <max_batch> <max_delay> <cap> -> run the sim for a burst of N arrivals at t=0, print result
Closes TWO mesh axes: dynamic-batching AND request-queue-admission. NO fake greens: the sim + neg-controls prove a
batch NEVER exceeds max_batch, an empty batch is NEVER dispatched, and admitted NEVER exceeds cap. license_tier: ORIGINAL
dependencies 2 imports · 0 importers
imports: nx_syscalls.nxnx_runtime.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| none |
functions
| 19 | func mb_w(fd: i64, s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(fd, s, n); return 0 } |
| 20 | func mb_wn(fd: i64, v: i64) -> i64 |
| 33 | func mb_p(s: *u8) -> i64 { return mb_w(1, s) } |
| 34 | func mb_pn(v: i64) -> i64 { return mb_wn(1, v) } |
| 35 | func mb_atoi(s: *u8) -> i64 { var v: i64 = 0; var i: i64 = 0; while s[i] != (0 as u8) { let c: i64 = s[i] as i64; if c >= 48 { if c <= 57 { v = v * 10 + (c - 48) } } i = i + 1 } return v } called by 1: main |
| 36 | func mb_min(a: i64, b: i64) -> i64 { if a < b { return a } return b } called by 1: mb_batch_take |
| 37 | func mb_streq(a: *u8, b: *u8) -> i64 { var i: i64 = 0; while b[i] != (0 as u8) { if a[i] != b[i] { return 0 } i = i + 1 } if a[i] != (0 as u8) { return 0 } return 1 } called by 1: main |
| 41 | func mb_admit(qlen: i64, cap: i64) -> i64 { if qlen < cap { return 1 } return 0 } |
| 43 | func mb_batch_ready(qlen: i64, max_batch: i64, waited_ms: i64, max_delay_ms: i64) -> i64 |
| 50 | func mb_batch_take(qlen: i64, max_batch: i64) -> i64 { return mb_min(qlen, max_batch) } |
| 54 | func mb_sim(arrivals: *i64, horizon: i64, max_batch: i64, max_delay: i64, cap: i64, outbox: *i64) -> i64 |
| 96 | func mb_gate() -> i64 |
| 172 | func main(argc: i64, argv: *i64) -> i64 |