code wiki / _hdl_build / nx_parallel_ingest.nx
nx_parallel_ingest.nx
buildroot/runtime/_hdl_build/nx_parallel_ingest.nx
about
nx_parallel_ingest.nx -- ingest many exceeds/capabilities IN PARALLEL without running out
of resources. The mechanism is three guarantees:
1. BOUNDED CONCURRENCY -- a worker pool of at most K live forks at any instant (never a
fork-bomb); items stream through the pool, K busy at a time, the rest queued.
2. PROCESS ISOLATION -- each worker is a fork; its scratch memory lives in ITS address
space and is reclaimed by the kernel on exit, so the parent's memory never grows with
the work. Per-item RAM is O(1) in the parent, not O(n).
3. COMPACT SHARED BANK -- results are fixed-size records in ONE shared mmap (MAP_SHARED,
0x21, so children write what the parent reads); the bank is O(n) bytes, not O(n) machine code.
So total resource use = K*(worker RAM) + n*(record size), both bounded and tunable -- you
can ingest thousands of exceeds with a handful of cores and a few KB. license_tier: ORIGINAL
dependencies 2 imports · 1 importers
imports: nx_syscalls.nxnx_boolsynth.nx
imported by: nx_parallel_ingest_test.nx
structs
| none |
consts
| none |
functions
| 17 | func par_mmap_shared(size: i64) -> *u8 called by 1: pi_main |
| 24 | func par_work(i: i64) -> i64 |
| 35 | func par_ingest(n: i64, K: i64, bank: *u8) -> i64 |