nx_web_shard_compact.nx

nx_web_shard_compact.nx -- SCALE compaction for big multi-segment shards (the web shard's rung). ss_compact/ss_compact_cap have EXACT semantics (last entry per key wins, tombstones kept, retired segments archived then an atomic manifest swap) but their key dedup is a LINEAR table scan = O(entries^2): fine at 4k keys, unusable at the web shard's ~1.4M entries (bulk docs + pr: priors). This organ is the same merge with an open-addressing HASH dedup = O(entries), plus a post-swap verification pass. Semantics kept byte-equal to ss_compact: chronological walk, last wins, ALL kinds carried (incl. kind-2 tombstones), archive-before-swap, every failure path returns BEFORE the swap. BYTE-BUDGETED SUFFIX FOLD (2026-07-29): an unbounded whole-shard merge materializes the merged segment in anonymous RAM (the writer buffer, plus aux blobs and key tables that all scale with merged bytes) -- the OOM killer reaped it twice at bytes-in=1883885124 on 2026-07-25, mid-merge, before the swap (never-brick held; shard untouched). The merge therefore now folds the LONGEST NEWEST contiguous manifest suffix whose .docs bytes fit a budget derived from the box itself (MemAvailable/WSC_BUDGET_SHARE; argv[2] in MB overrides). Replacing a contiguous suffix with its fold preserves last-wins semantics EXACTLY: the run is merged in chronological order, kind-2 tombstones are carried, and older (kept) segments remain shadowed by the merged tail precisely as they were by the unmerged tail. Repeated runs converge the shard LSM-style with bounded memory; a streaming (file-backed) writer that lifts the budget entirely is the named next rung. usage: nx_web_shard_compact <domain> [budget-MB] license_tier: ORIGINAL