nx_url_canon.nx
buildroot/runtime/nx_url_canon.nx
about
nx_url_canon.nx -- HTML attribute -> WIRE URL canonicaliser. LIB (no main, no imports).
module: nishi-core.search.urlcanon
capability: CORE_COMPUTE
WHY THIS FILE EXISTS AS A FILE: the same defect was found in two unrelated harvesters on the same
day -- the crawler's outlink harvest (debt 1786031857: arxiv + loc.gov urls carrying a literal
`&`) and the media ledger (37 of 438 image urls, same shape). Both had hand-rolled attribute
copying. Rather than fix it twice and wait for the third, the canonicaliser is ONE definition that
every harvester imports. The two RESOLVERS (olh_resolve_root, nx_url_resolve) stay separate on
purpose -- merging resolvers to deduplicate is how live capabilities were deleted here before.
⚠DO NOT MOVE THIS INTO nx_url_resolve. That function also resolves HTTP `Location:` headers, which
are not HTML: an `&` in a Location header is five literal bytes of the real url, and decoding
it would corrupt a correct redirect.
★DECODE BELONGS AT THE LAYER THAT KNOWS THE BYTES CAME FROM HTML, NOT AT THE LAYER THAT ONLY
KNOWS THEY ARE A URL.
license_tier: ORIGINAL
dependencies 0 imports · 3 importers
imports: none
imported by: nx_html_extract_imgs.nxnx_html_extract_links.nxnx_outlink_harvest.nx
structs
| none |
consts
| 20 | const UC_SCRATCH: i64 = 4096 // per-call canon scratch; a url longer than this falls back to raw bytes |
| 208 | const UC_HTTPS_UPGRADE: i64 = 1 |
| 209 | const UC_CH_FRAG: i64 = 35 |
functions
| 51 | func uc_hexdig(v: i64) -> i64 called by 1: uc_emit_byte |
| 56 | func uc_url_unsafe(c: i64) -> i64 called by 1: uc_emit_byte |
| 71 | func uc_emit_byte(c: i64, dst: *u8, dp: i64, cap: i64) -> i64 |
| 84 | func uc_emit_cp(cp: i64, dst: *u8, dp: i64, cap: i64) -> i64 |
| 107 | func uc_ent_eq(src: *u8, off: i64, n: i64, lit: *u8) -> i64 called by 1: uc_named_cp |
| 118 | func uc_named_cp(src: *u8, off: i64, n: i64) -> i64 |
| 151 | func uc_numeric_cp(src: *u8, off: i64, n: i64) -> i64 called by 1: nx_url_canon |
| 211 | func uc_lc_ascii(c: i64) -> i64 { if c >= 0x41 { if c <= 0x5A { return c + 0x20 } } return c } called by 1: uc_wire_norm |
| 217 | func uc_wire_norm(b: *u8, n: i64, cap: i64) -> i64 |
| 281 | func nx_url_canon(src: *u8, srclen: i64, dst: *u8, dstcap: i64) -> i64 |