code wiki / _hdl_build / nx_pagerank_build.nx

nx_pagerank_build.nx

buildroot/runtime/_hdl_build/nx_pagerank_build.nx

11307 B213 linesdepth 5pulls 12 transitivereach 1 importersview sourcekind tooltopic pagerank
docsdependenciesstructsconstsfunctions

about

nx_pagerank_build.nx -- P1 STEP 2: build the web LINK GRAPH from out:<cid> edge rows (captured by nx_web_crawl_step's outlink capture) and compute PageRank authority, storing pr:<cid> priors back into the web shard. Node identity = cid(url) (the frontier/outlink hash) -> an out: key's cid is the source node, its packed-i64 value are the target nodes. Interns cids to dense indices, runs pr_compute (nx_pagerank), stores pr:<cid> = rank. Serve-time authority (step 3) = content_cid -> url:<cid> -> ci_hash(url) -> pr:<cid>. usage: nx_pagerank_build [domain=web] [iters=100] license_tier: ORIGINAL

dependencies 3 imports · 1 importers

nx_corpus_ingest.nx nx_itoa_lib.nx nx_pagerank.nx nx_pagerank_build.nx nx_pagerank_build_gate.nx

imports: nx_corpus_ingest.nxnx_itoa_lib.nxnx_pagerank.nx

imported by: nx_pagerank_build_gate.nx

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main dss_prefix pb_puts pb_run ss_open2 sys_mmap ss_manifest_dyn ss_manifest_file_dyn sys_mmap ↻ ss_cat ss_readall sys_openat_rd sys_lseek sys_mmap ↻ sys_read sys_close sys_munmap ss_cat ↻ ss_loadfile sys_map_file sys_openat_rd ↻ sys_lseek ↻ sys_close ↻ ss_readall ↻ ss_load_aux2 sys_mmap ↻ ss_cat ↻ ss_loadfile ↻ ss_r32 sys_munmap ↻ ssl_total_keys ss_r32 ↻ ssl_pow2 ssl_build ss_r32 ↻ ssl_hash_entry ss_r32 ↻ ssl_key_eq ss_r32 ↻ ss_r32 ↻

structs

none

consts

11const PB_MAGIC_16777216: i64 = 16777216
12const PB_MAGIC_1000000: i64 = 1000000
17const PB_MAXNODE: i64 = 16777216 // distinct nodes per run (16M; one ~40k-page WARC links ~1-4M targets)
18const PB_MAXEDGE: i64 = 33554432 // directed edges per run (32M)
19const PB_HSIZE: i64 = 33554432 // hash slots = 2^25 (2x MAXNODE load-factor headroom)
20const PB_HSHIFT: i64 = 38 // take the TOP 25 bits of the 63-bit mixed hash (63 - 25)
21const PB_HMUL: i64 = 7046029254386353131 // odd 63-bit multiplicative-mix constant (Fibonacci-hash family)

functions

23func pb_puts(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 }
called by 3: pb_collectpb_runmain
28func pb_num(v: i64) -> i64 { nxi_out(v); return 0 }
called by 2: pb_runmain calls 1: nxi_out
30func pb_key_cid(k: *u8, kl: i64) -> i64
called by 1: pb_collect
36func pb_prkey(cid: i64, out: *u8) -> i64
called by 2: pb_rungp_pr
51func pb_hslot(cid: i64) -> i64
called by 2: pb_internpb_seen
58func pb_intern(cid: i64, cidtab: *i64, idxtab: *i64, node: *i64, nn: *i64) -> i64
called by 1: pb_collect calls 1: pb_hslot
79func pb_seen(cid: i64, seentab: *i64) -> i64
called by 1: pb_collect calls 1: pb_hslot
93func pb_collect(h: *i64, cidtab: *i64, idxtab: *i64, seentab: *i64, node: *i64, nn: *i64, efrom: *i64, eto: *i64, srcflag: *u8) -> i64
called by 1: pb_run calls 6: ss_r32pb_key_cidpb_seenss_hgetpb_internpb_puts
148func pb_run(prefix: *u8, iters: i64, segid0: i64) -> i64
called by 2: mainmain calls 9: ss_open2pb_putspb_numpb_collectpr_computess_begin_cap+3
198func main(argc: i64, argv: *i64) -> i64
calls 4: dss_prefixpb_putspb_runpb_num