nx_wpt_ingest.nx

nx_wpt_ingest.nx -- BR21: byte-preserving WPT corpus ingest. Carries wsp_ingest_dir. ONE CALL: nx_wpt_ingest <wpt-dir-path> <40-hex-commit> <dest-dir> nx_wpt_ingest dom/nodes a7b5671e50ee3610ec3ad2e1278a33b2cb11339c bench/wpt/dom_nodes WHAT IT DOES, AND WHY EACH STEP IS THERE: 1. REFUSES a commit that is not 40 lowercase hex. A branch name resolves to whatever HEAD happens to be, and a corpus acquired from a moving reference is reproducible by nobody -- which is the whole defect BR21 exists to close. A pin that is not a pin is worse than no pin, because the manifest would still record it and read as authoritative. 2. Fetches the directory manifest ONCE from GitHub's contents API, at the pin. That is one request against the API's 60/hour unauthenticated quota per DIRECTORY, not per file; every body then comes from raw.githubusercontent.com, which is not on that quota. 3. For each declared .html file: if it is already on disk AND satisfies both declared numbers, it is REUSED and not re-fetched (the ingest is idempotent). Otherwise it is fetched into a reserve DERIVED FROM ITS OWN DECLARED SIZE, verified against BOTH the declared size and the declared git blob sha, written as RAW BYTES with an explicit length, and READ BACK AND COMPARED BYTE FOR BYTE. 4. Writes MANIFEST.tsv carrying the pin, and writes or REMOVES the INCOMPLETE-DO-NOT-PUBLISH marker so the directory's publishability is a fact on disk rather than a memory. THE TEXT LAYER IS THE DEFECT AND THERE IS NONE HERE. Every content path in wa_ carries an explicit length. The transport hands back a body and a byte count; nothing between the socket and the file looks for a NUL. That is what makes html/syntax/parsing-html-fragments reachable at all -- its UTF-16LE-with-BOM fixture has a NUL as its fourth byte, and any C-string carrier truncates it there and reports success. license_tier: ORIGINAL