code wiki / _hdl_build / nx_bulk_index.nx

nx_bulk_index.nx

buildroot/runtime/_hdl_build/nx_bulk_index.nx

8970 B142 linesdepth 21pulls 174 transitivereach 0 importersview sourcekind tool
docsdependenciesstructsconstsfunctions

about

nx_bulk_index.nx -- TARGETED bulk indexer (operator 2026-07-06: "we should have thousands of results ... index the individual works, not 3 search pages"). Reads a URL list (one per line), fetches each over the best-effort stack, PER-HOST paces (nx_crawl_pace -- don't get banned), extracts (nx_html_to_text + nx_struct_extract title/desc), and indexes each into dp-web-pub- with ONE shard handle + batched commits. This is nx_url_index's core looped -- the efficient path to index a performer's whole catalog. usage: nx_bulk_index <url-list-file> license_tier: ORIGINAL

dependencies 7 imports · 0 importers

nx_corpus_ingest.nx nx_struct_extract.nx nx_js_eval.nx nx_crawl_pace.nx nx_x509_trust_store.nx nx_trust_store_load_from_certdata. nx_https_fetch_follow.nx nx_bulk_index.nx

imports: nx_corpus_ingest.nxnx_struct_extract.nxnx_js_eval.nxnx_crawl_pace.nxnx_x509_trust_store.nxnx_trust_store_load_from_certdata.nxnx_https_fetch_follow.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main bw nx_trust_store_load_from_c sys_mmap sys_read_file sys_openat_rd sys_lseek sys_mmap ↻ sys_read sys_close nx_nss_certdata_parse sys_mmap ↻ _pat_class_cert _pat_value_octal _pat_end _find_newline _starts_with _parse_octal_line sys_mmap ↻ _is_space _decode_octal_escape _is_octal trust_store_alloc sys_mmap ↻ nx_x509_trust_store_load sys_mmap ↻ x509_parse sys_mmap ↻ asn1_cursor_init asn1_expect_tag sys_mmap ↻ asn1_read_tlv_header asn1_read_tag asn1_read_length asn1_read_length ↻ x509_read_tlv sys_mmap ↻ asn1_expect_tag ↻ x509_read_alg_id sys_mmap ↻

structs

none

consts

15const K_MAGIC_4194304: i64 = 4194304
16const K_MAGIC_4096: i64 = 4096
17const K_MAGIC_1048576: i64 = 1048576
18const K_MAGIC_8192: i64 = 8192
19const K_MAGIC_1024: i64 = 1024
20const K_MAGIC_2048: i64 = 2048
21const K_MAGIC_4095: i64 = 4095
22const K_MAGIC_8190: i64 = 8190

functions

24func bw(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(1,s,n); return 0 }
called by 1: main
25func bn(v: i64) -> i64 { if v==0 { sys_write(1,"0" as *u8,1); return 0 } var m: i64=v; if m<0 { sys_write(1,"-" as *u8,1); m=0-m } let t: *u8=sys_mmap(24); var k: i64=0; while m>0 { t[k]=(48+(m%10)) as u8; m=m/10; k=k+1 } let o: *u8=sys_mmap(24); var q: i64=k-1; var x: i64=0; while q>=0 { o[x]=t[q]; x=x+1; q=q-1 } sys_write(1,o,x); return 0 }
called by 1: main
26func bslen(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} return n }
called by 1: main
27func bhas(hay: *u8, hn: i64, ndl: *u8) -> i64 { var nl: i64=0; while ndl[nl]!=(0 as u8){nl=nl+1} var i: i64=0; while i+nl<=hn { var m: i64=1; var j: i64=0; while j<nl { if hay[i+j]!=ndl[j] { m=0; j=nl } else { j=j+1 } } if m==1 { return 1 } i=i+1 } return 0 }
called by 1: main
28func bhost(url: *u8, out: *u8) -> i64
called by 1: main
37func main(argc: i64, argv: *i64) -> i64
calls 21: bwnx_trust_store_load_from_certdatasys_read_filebndss_prefixss_manifest_cap+15