code wiki / warc
topic: warc
4 modules sharing the warc name family (derived from the tree's prefix discipline).
| module | description | lines | funcs |
|---|---|---|---|
| nx_warc_index.nx | C3: the WARC -> INVERTED-INDEX ingest pipeline = the heart of "ingest Common Crawl into our | 94 | 9 |
| nx_warc_index_gate.nx | proves the C3 ingest pipeline: WARC 'response' records -> extract content -> index -> | 79 | 6 |
| nx_warc_reader.nx | LIB: sovereign WARC (ISO 28500 / Common Crawl) record reader. A .warc file is a sequence of | 101 | 8 |
| nx_warc_reader_gate.nx | GREEN/RED gate for nx_warc_reader. Builds a SYNTHETIC plain-WARC fixture in memory (a | 100 | 7 |