nx_bright_prep.nx

nx_bright_prep.nx -- BRIGHT (huggingface xlangai/BRIGHT, cc-by-4.0) reshaped into the BEIR layout nx_beir_eval reads (search R0d be_bright). usage: nx_bright_prep <split> <examples.parquet> <documents.parquet> <outdir> Writes, through the sovereign parquet reader (nx_parquet_lib): <outdir>/corpus.tsv doc id TAB content (the documents file, id and content, via pq_tsv) <outdir>/queries.tsv query id TAB query (the examples file) <outdir>/qrels/test.tsv the BEIR header line, then query id TAB doc id TAB 1 for every gold_ids element <outdir>/excluded.tsv query id TAB doc id for every excluded_ids element: BRIGHT lists the query's own source documents there, so a retriever must not be credited for finding them <outdir>/../LICENSE the dataset, its source and its licence beside the bytes (a data asset carries its licence in the same directory, the estate's rule for foreign data) Every file is staged as .tmp and renamed into place. Tabs, newlines and carriage returns inside a cell become spaces. A row with a null id or query is COUNTED and skipped; a gold or excluded element whose row has no id is COUNTED as an orphan. The known-answer test is the dataset card's num_examples per split (search.refs brightds26): queries must equal it and corpus rows must equal the documents count. The work lives in bp_run so the gate drives it in-process; main prints the receipt. Exit 0 OK, 1 REFUSED, 2 usage. Measured 2026-09-14 on eleven splits: every count agrees with the card, orphans 0 (search.plan 1789433620). license_tier: ORIGINAL No hw writes (Rule 26).