nx_mvault_walk.nx

nx_mvault_walk.nx -- the LIVE reindex DRIVER, VW02 (MV-1 bulk-migration engine). VW01 was a single-level, full-read walk: it SKIPPED >512MiB files (seq119 OOM guard) -- i.e. most cam recordings -- and its one-reg_put-per-item store path would have both crawled (a new segment per put) and corrupted the ids index past ~15k items (fixed 1MiB index buffer). VW02 is the measured rebuild: * RECURSIVE: iterative BFS dir queue, ONE shared getdents buffer. * STREAMING CID: every file hashes in chunks (the head window doubles as the sniff bytes) -- nothing is skipped for size, RAM is O(1) per file. * IN-MEMORY DEDUP SETS seeded once from mv:ids / p:ids -- zero per-file store reads; BATCHED commits (one segment per ~24MiB of records, not one per item) -- the seg-store quadratic + index-rewrite blowup is dodged. * PATH-RESUME PLANE knowledge/store/mvaultpath-: key p:<cid of "path|size">, val = content CID. Re-runs skip already-hashed unchanged paths (daily DVR ingest = incremental by design). Size-keyed, not mtime-keyed: recordings are append-once; an in-place same-size rewrite would be missed (declared). * BG MODE (`bg <logfile>`): forks a detached worker; fds 0/1/2 close so the tools-daemon capture pipe EOFs immediately; progress + final JSON append to <logfile>. TB-scale hashing runs for hours -- a synchronous MCP call drops its response at ~15s. * POLICY FILTERS (counted, never silent): dot-names skipped (.trickplay sidecar dirs), *-poster.jpg sidecars skipped, 0-byte files skipped (hard-crash torn artifacts must not become vault records). * selftest: KAT-anchored teeth, emits verdict=GREEN|RED (D001 anchor). RECORD-ONLY = NON-MUTATING BY CONSTRUCTION (unchanged): never moves a byte; only the seg-store index planes are written (and only on commit). ENVELOPE (declared): set ~950k cids / queue 16MiB dirs / ids blob 64MiB -- overflow sets fatal=1 in the output, never a silent cap. nx_mvault_walk <dir> <gen|real> <source> [commit|dry] [bg <logfile>] nx_mvault_walk selftest license_tier: ORIGINAL No hw writes (Rule 26).