nx_store_janitor.nx

nx_store_janitor.nx -- reclaim SUPERSEDED seg-store segment files. The janitor pass that seq724/RK013 named as reclaimable and nobody had built. MEASURED 2026-07-30 BEFORE WRITING THIS: knowledge/store holds 63,669 MB across 14,585 segment .docs files, while all 1,088 manifests TOGETHER reference only 3,031 live segments. debt-manifest.txt lists exactly TWO segments while 2,055 debt-seg-*.docs exist on disk, ~2.4 GB for one plane. So roughly 79 percent of the information plane's bytes are segments no reader can ever reach: the seg-store is append-only and every commit writes a NEW segment, so superseded generations simply accumulate forever. WHY THIS IS SAFE BY CONSTRUCTION: the manifest IS the definition of live. ss_manifest/ss_scan_seglist read ONLY segments listed there, so a file whose segid is absent from its own plane's manifest is unreachable BY DEFINITION, not by inference. That is the whole safety argument and it is why this tool refuses to run when it cannot read the manifest -- an unknown live set means an unknown safe set. RULE 13 -- MOVE ASIDE, NEVER DELETE. Superseded files are RENAMED into knowledge/store/retired/ (the convention already present on the box). Nothing is unlinked, so a mistake is reversible by renaming back. DRY RUN IS THE DEFAULT: it reports what it WOULD move and touches nothing. Pass 'apply' to actually move. 2026-08-18 -- THE BIRTH RACE, MEASURED TWICE AND CLOSED BY CONSTRUCTION. ss_commit_body writes the four segment files (.docs/.idx/.pos/.imp) FIRST and appends the manifest line SECOND, and every new segid is max(manifest)+1 under the plane lock. So a segment being born is, for a moment, an unlisted file set with an id ABOVE the manifest's max. This janitor read the manifest, then walked, and moved exactly such a birth (comparewatch-seg-1786828803 on 08-16/17, debt-seg-1787064612 on 08-18); its post-sweep race check then put back ONLY the .docs, so every reader saw a live segment with no index: ss_get still answered q:n while sts_load resolved zero rows -> "lossy load"/SHORTFALL refusals on the estate's debt board and hive plane, and one sibling seat's debt row was overwritten during the hand repair. Two fixes, both here: 1. AN UNLISTED SEGMENT WHOSE ID IS ABOVE THE MANIFEST'S MAX IS A BIRTH IN FLIGHT, NEVER GARBAGE. It is counted (inflight_files=, in files like every other counter on the line) and left alone; once the plane's max passes it, it is either listed (live) or a crashed birth (swept on a later beat). No lock is needed: the ordering is the store's own commit protocol. 2. THE POST-SWEEP RESTORE PUTS BACK EVERY FILE OF A LIVE SEGMENT, not the record alone: it walks retired/ and renames back each <base>seg-<id>.<ext> whose id is live NOW and whose live path is absent -- no extension list here (the store owns its layout), and never a clobber. The same routine is the `heal` verb, so the two hand repairs above become one call: nx_store_janitor <prefix> heal. And the manifest is read whole via sys_read_file -- the old fixed 256 KiB read plus an 8192-line table were SILENT CAPS: a manifest past either would have swept live segments as dead. EXIT: 0 clean/reported - 2 usage - 3 REFUSED (manifest unreadable or empty live set) - 4 nothing superseded license_tier: ORIGINAL No hw writes (Rule 26).