nx_crawl_doc.nx

nx_crawl_doc.nx -- per-page processing unit for the sovereign crawler. module: nishi-core.search.crawl_doc depends: nx_html_to_text.nx, nx_simhash.nx, nx_search_inverted.nx capability: CORE_COMPUTE wired_status: FULLY_WIRED The back half of the crawl loop (the half that is design-sensitive and SEO-resistant), composing already-proven sovereign modules: fetched HTML --nx_html_to_text--> visible text (tags/script/style stripped) --nx_simhash--------> 64-bit content fingerprint The crawler then uses the fingerprint for content near-dup / novelty BEFORE indexing, so a scraped copy never pollutes the index while unique content is kept -- demotion EARNED from measured content, never a domain/title prejudice (operator cardinal 2026-05-29). Front half (resolve + fetch over our own DNS/TLS/HTTP) runs in the operator's networked environment.