nx_paced_fetch.nx

nx_paced_fetch.nx -- THE ONE PACING HOOK FOR THE CAPTURE PATH (/compare/mediaingest R0; the vault's watch contract symbol is vc_pace_hook in nx_vault_capture, which delegates here so every capture verb shares one hook). WHY (measured 2026-08-30, corpus_complete=1): nx_crawl_pace -- Retry-After honoured, exponential backoff, decay on success, a table persisted across processes -- was imported by the crawl lane and nx_4chan and by NONE of the capture verbs (nx_vault_capture, nx_mvault_fetch, nx_hls_get, nx_media_grab), so every capture hit a host as fast as its loop ran. That is the rate-limit the operator meets on Harvestr, unmanaged here by construction. THIS IS NOT A SECOND PACING RULER. Every decision below is nx_crawl_pace's own arithmetic (pace_*_tbl). The lib adds only what the capture verbs lacked: URL to host, the before/after pair around ONE fetch, a BURST form for segment streams, and ONE process-lifetime table buffer. Spend-gate 2026-08-30 listed the incumbents and they are policies, not this hook: nx_retry_policy (per-call retry maths) and the builder-generated _cn_fetch/_pe_crawl/ _wb_pace governors (per-organ min-gap copies -- the duplicate-ruler class this composition exists to stop). THE BURST FORM (pf_before_burst): segments of ONE stream are not separate acts of politeness. The default interval is PACE_BASE_MS = 1 s per host, which on a 1000-segment HLS VOD is a 17-minute floor for no reason a host cares about. The burst form pays the wait ONLY once the host has actively throttled us (pace_should_defer: consec_throttle > 0, or a published Crawl-delay longer than one fetch can sleep through) -- the exact signal the pacer's own contract names as the only one that means refused-us. A healthy CDN streams at full speed; a 429 still lands as backoff on the very next segment. pf_after is called on EVERY response either way, so the table always carries the truth. MEMORY: pace_before/pace_after mmap a 160 KB table per call and never unmap it (nx_crawl_pace R13b measured that for the crawler). Called once per FETCH by a 1000-segment download that is ~320 MB of touched anon pages. This lib owns ONE buffer for the process and reloads it IN PLACE (pace_load_tbl_into) before every decision, so a write by another process, or by our own pf_after, is always seen and nothing leaks. license_tier: ORIGINAL