nx_batched_tput.nx
buildroot/runtime/nx_batched_tput.nx
about
nx_batched_tput.nx -- AGGREGATE THROUGHPUT of batched multi-sequence
decode (the server axis). Single-stream decode is at its CPU floor
(~15 tok/s); batching reads the 209MB weights ONCE per round for M
sequences, so aggregate tok/s should scale ~M until compute catches
memory. Prefill ONE prompt, fork M ways (prefix-shared), run T batched
v4b rounds, report aggregate M*T tokens / wall. Sweep M=1,2,4,8. This
is the honest "how fast can the sovereign engine actually serve" number.
expect_exit: 0 license_tier: ORIGINAL
dependencies 25 imports · 0 importers
diagram shows first 10 each side; +15 more imports, +0 more importers in the complete lists below.
imports: nx_syscalls.nxnx_tier.nxnx_le.nxnx_bpe.nxnx_gguf.nxnx_gguf_load.nxnx_gguf_meta.nxnx_f32.nxnx_f32_kv_cache.nxnx_f32_lazy_weight.nxnx_f32_llama_block.nxnx_f32_llama_block_v4.nxnx_f32_llama_stack_v4.nxnx_f32_llama_layer_lazy_load.nxnx_f32_llm.nxnx_f32_llm_v4.nxnx_f32_llm_read_dims.nxnx_f32_bpe_load.nxnx_f32_llm_special_tokens.nxnx_f32_sampler.nxnx_reasoning.nxnx_kvcache.nxnx_f32_attn_paged.nxnx_f32_llama_v4p.nxnx_f32_llama_v4b.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 34 | const BT_MAGIC_1000000000: i64 = 1000000000 |
| 35 | const BT_MAGIC_67108864: i64 = 67108864 |
| 36 | const BT_MAGIC_262144: i64 = 262144 |
| 37 | const BT_MAGIC_524288: i64 = 524288 |
| 38 | const BT_MAGIC_151644: i64 = 151644 |
| 39 | const BT_MAGIC_151645: i64 = 151645 |
| 41 | const BT_T: nx_int = 16 // decode rounds (tokens/seq) per measurement |
functions
| 43 | func bt_w(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 } |
| 44 | func bt_wn(v: i64) -> i64 |
| 59 | func bt_measure(rc: *NxReasonCfg, pool: *NxPagedPool, seq0: *NxPagedSeq, |
| 98 | func main() -> i64 |