code wiki / (root) / nx_inflate.nx

nx_inflate.nx

buildroot/runtime/nx_inflate.nx

16119 B348 linesdepth 2pulls 2 transitivereach 4 importersview sourcekind librarytopic inflate
docsdependenciesstructsconstsfunctions

about

nx_inflate.nx -- SOVEREIGN DEFLATE DECOMPRESSION (RFC 1951) + gzip (1952) + zlib (1950). WHY THIS EXISTS. Our sovereign HTTPS client could REQUEST gzip and never inflate it. That gap cost us twice: db.bepis.moe and most of the modern web returned bodies we could not read, and worse, the search lane once tokenized COMPRESSED BYTES AS TEXT and poisoned its own index (debt 1785794199) -- a decompressor that is missing does not fail loudly, it corrupts quietly. WHAT WAS ALREADY HERE, AND WHY THIS IS NOT A REBUILD. `_inflate_lib_authored.nx` (builder- generated) already had a structurally correct canonical-Huffman inflate: bit reader, code-length ordering, length/distance tables. That algorithm shape is REUSED here. What it lacked is exactly what makes a decompressor safe to point at the internet: - NO OUTPUT CAPACITY. `out: *u8` with no bound. A hostile or merely large body writes past the allocation -- a heap overflow driven by untrusted network input. - NO DISTANCE VALIDATION. `out[outpos-dist]` with dist > outpos reads BEFORE the buffer. - NO CONTAINER. Raw DEFLATE only, so HTTP Content-Encoding: gzip was still unreadable. - NO INTEGRITY CHECK. gzip carries CRC32 + ISIZE precisely so corruption is detectable. - A DECODE FAILURE READ AS "DONE". An invalid symbol returned -1 and the caller treated it as end-of-block, so a corrupt stream SILENTLY TRUNCATED instead of erroring -- the same quiet-corruption shape as the missing decompressor itself. ★★CORRECTION 2026-08-04 (retraction, one day after shipping): THE PREMISE ABOVE WAS INCOMPLETE. The estate ALREADY HELD a complete, capacity-bounded, CRC-verified gzip inflate: nx_gzip_wrap.nx (nx_gzip_inflate, KAT-proven against real python-gzip streams, ~10 consumers incl the browser, nx_page_ingest, nx_research_engine, the CommonCrawl ingest, nx_libdata) plus nx_zlib_wrap.nx for RFC 1950. The corpus-ask before this build surfaced only _inflate_lib_authored.nx and missed both wraps (they are named *_wrap, the search was too narrow). THIS FILE IS THEREFORE A SECOND IMPLEMENTATION of a capability the estate held. The fetch lane was wired to the INCUMBENT (nx_https_get_cli2 -> nx_gzip_inflate/nx_zlib_inflate, 2026-08-04) so this duplicate gains no consumers; its current consumers are nx_gunzip (CLI) + nx_inflate_gate only. What it has that the incumbent lacks: a raw RFC-1951 entry point (no container) + system-gzip cross-impl gate fixtures. CONVERGENCE DEBT 1785854636: fold those into nx_gzip_wrap and retire this file, or demote it to gate-oracle-only. DO NOT ADD NEW CONSUMERS. ★A DECOMPRESSOR IS A PARSER POINTED AT HOSTILE INPUT: every read bounded, every write bounded, every failure LOUD. Truncating quietly is the one behaviour worse than refusing. ERRORS (all distinct, all negative -- a caller can tell WHICH wall fired): -1 output would exceed capacity -2 input exhausted / truncated stream -3 invalid huffman symbol -4 back-reference distance before buffer start -5 bad block type (BTYPE=3) -6 stored-block LEN/NLEN mismatch -7 not a gzip container -8 unsupported gzip method/flags

dependencies 1 imports · 4 importers

nx_syscalls.nx nx_inflate.nx nx_entity_media.nx nx_entity_seeds.nx nx_gunzip.nx nx_inflate_gate.nx

imports: nx_syscalls.nx

imported by: nx_entity_media.nxnx_entity_seeds.nxnx_gunzip.nxnx_inflate_gate.nx

structs

none

consts

45const INF_MAGIC_1025: i64 = 1025
46const INF_MAGIC_1537: i64 = 1537
47const INF_MAGIC_2049: i64 = 2049
48const INF_MAGIC_3073: i64 = 3073
49const INF_MAGIC_4097: i64 = 4097
50const INF_MAGIC_6145: i64 = 6145
51const INF_MAGIC_8193: i64 = 8193
52const INF_MAGIC_12289: i64 = 12289
53const INF_MAGIC_16385: i64 = 16385
54const INF_MAGIC_24577: i64 = 24577
56const INF_CRC_POLY: i64 = 0xEDB88320
57const INF_M32: i64 = 0xFFFFFFFF

functions

60func inf_crc32(buf: *u8, n: i64) -> i64
called by 3: maininf_gunzipmain
78func inf_bits(buf: *u8, len: i64, cur: *i64, n: i64) -> i64
93func inf_build(ln: *i64, n: i64, ct: *i64, sy: *i64, off: *i64) -> i64
called by 1: inf_raw
109func inf_decode(buf: *u8, len: i64, cur: *i64, ct: *i64, sy: *i64) -> i64
called by 2: inf_codesinf_raw calls 1: inf_bits
128func inf_codes(src: *u8, sl: i64, cur: *i64, out: *u8, outcap: i64, st: *i64,
called by 1: inf_raw calls 2: inf_decodeinf_bits
168func inf_raw(src: *u8, srclen: i64, out: *u8, outcap: i64) -> i64
303func inf_gunzip(src: *u8, srclen: i64, out: *u8, outcap: i64) -> i64
339func inf_zlib(src: *u8, srclen: i64, out: *u8, outcap: i64) -> i64
called by 1: main calls 1: inf_raw