code wiki / _hdl_build / nx_search_census.nx

nx_search_census.nx

buildroot/runtime/_hdl_build/nx_search_census.nx

30185 B534 linesdepth 22pulls 200 transitivereach 1 importersview sourcekind tooltopic search
docsdependenciesstructsconstsfunctions

about

nx_search_census.nx -- G1: THE SEARCH CORPUS, THROUGHPUT AND DIVERSITY CENSUS. /compare/search rung G1, contracts idx_corpus_census and wc_throughput_census. WHY IT EXISTS. On 2026-09-03 the operator could see that the index was not growing and the BOARD COULD NOT. /compare/search carried four latency rows and a relevance row and NOTHING for corpus size, crawl throughput or host diversity; the one row that touched scale had already withheld its Nishi figure as "prose, never a measurement" and nobody had measured it since. Nothing else in the estate measured it either -- knowledge/status/growth_series.log tracks the PM board, debts and maturity and carries no corpus field at all -- so the one axis the operator was complaining about was invisible to every consumer and therefore priced at zero by the ranker. WHAT IT REFUSES TO DO. It prints LEVELS and never a rate. A rate needs two samples in time and the estate ALREADY has a time spine, nx_growth_series, whose report verb computes d/w/m deltas per axis. A second spine here would be the duplicate-ruler defect, so this organ is a PRODUCER for that one: it TRUNCATE-writes a canonical status file and lets the spine carry the trajectory. The throughput axis is the DIFFERENCE between two spine rows, never a number this organ invents. EVERY COUNT CARRIES ITS COVERAGE, and that is not a slogan here. The key census is COMPLETE over the manifest and says so, with its partition summed and the sum printed. The outcome partition is a DECLARED TAIL WINDOW and says so. This organ was written in the same session a sibling validator was caught reading the OLDEST 3.2 percent of a 129 MB log while printing "sampling newest-first" -- a window published without its denominator reads as a population, and that is exactly how a three-week old fossil became evidence about today. ONE FAIRNESS RULER. The fair verdict is wc_frontier_fair, imported from the ENFORCER itself, and the host parser is the crawler's own wc_host. Re-deriving either here would be a second ruler that agrees today and diverges silently on the next edit to one side. DECLARED GAPS, NAMED RATHER THAN HIDDEN. (1) doc= counts seg-store ENTRIES, which are append-only, so it is an UPPER BOUND on distinct documents and the output says so on its own line; distinct-key dedup is the named next rung. (2) The fr: pending-versus-done split is NOT reported: splitting it needs an ss_hget per frontier key and the live shard holds millions, so the honest choices were a silent sample or a named absence, and this takes the absence. usage: nx_search_census [shard-prefix] [outcomes-log] license_tier: ORIGINAL expect_exit: 0

dependencies 2 imports · 1 importers

nx_web_crawl_step.nx nx_intlog.nx nx_search_census.nx nx_search_census_gate.nx

imports: nx_web_crawl_step.nxnx_intlog.nx

imported by: nx_search_census_gate.nx

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sc_w ss_open2 ss_open3 sys_now_us sys_mmap nxa_die sys_write sys_exit nxa_lock_take nxa_lock_addr sys_write ↻ nxa_lock_give nxa_lock_addr ↻ nxa_report_overrun sys_write ↻ nxa_dump_printable nxa_dump_sizes sys_clock_gettime_mono sys_mmap ↻ ss_manifest_dyn ss_manifest_file_dyn sys_mmap ↻ ss_cat ss_readall sys_openat_rd sys_lseek sys_mmap ↻ sys_read sys_close sys_munmap ss3_admit ss3_ann sys_mmap ↻ ss_cat ↻ ss_catn sys_mmap ↻ sys_write ↻ sys_munmap ↻ sys_now_us ↻

structs

none

consts

39const SC_SHARD: *u8 = "knowledge/store/dp-web-pub-"
40const SC_LOG: *u8 = "knowledge/status/crawl_outcomes.log"
41const SC_STATUS: *u8 = "knowledge/status/search_census.status"
44const SC_LOGCAP: i64 = 4194304
49const SC_HOSTSLOTS: i64 = 65536
50const SC_HOSTBUF: i64 = 64
51const SC_URLBUF: i64 = 4096
52const SC_STATBUF: i64 = 4096
53const SC_HASH_MUL: i64 = 1000003
54const SC_TAB: i64 = 9
55const SC_NL: i64 = 10
56const SC_PERMIL: i64 = 1000
249const SC_CROSSOVER_CONF: *u8 = "knowledge/search_crossover.conf"
250const SC_POS_EXT: *u8 = ".pos"
251const SC_POS_MAGIC_N: i64 = 78
252const SC_POS_MAGIC_X: i64 = 88
253const SC_POS_MAGIC_Q: i64 = 81
254const SC_POS_MAGIC_1: i64 = 49
255const SC_POS_HDR: i64 = 8
256const SC_CONF_VALCAP: i64 = 2048
257const SC_INTBUF: i64 = 64
258const SC_SPACE: i64 = 32
259const SC_CR: i64 = 13

functions

58func sc_w(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 }
called by 1: main
59func sc_n(v: i64) -> i64
called by 1: main
67func sc_cat(out: *u8, o: i64, s: *u8) -> i64 { var i: i64 = 0; while s[i] != (0 as u8) { out[o] = s[i]; o = o + 1; i = i + 1 } return o }
called by 2: maingsc_row
68func sc_catn(out: *u8, o: i64, v: i64) -> i64
called by 1: main
79func sc_lit_at(p: *u8, len: i64, lit: *u8) -> i64
89func sc_key_at(kb: *u8, eo: i64, kl: i64, lit: *u8) -> i64
called by 1: idx_corpus_census calls 1: sc_lit_at
97func sc_atoi_at(p: *u8, s: i64, e: i64) -> i64
103func sc_hash(p: *u8, n: i64) -> i64
called by 1: sc_host_new
111func sc_host_new(tbl: *i64, slots: i64, hp: *u8, hl: i64) -> i64
called by 1: wc_throughput_census calls 1: sc_hash
134func idx_corpus_census(h: *i64, out: *i64) -> i64
called by 1: main calls 2: ss_r32sc_key_at
176func wc_throughput_census(buf: *u8, n: i64, out: *i64, hosttbl: *i64, slots: i64, hbuf: *u8, ubuf: *u8) -> i64
called by 1: main calls 3: sc_atoi_atsc_lit_atsc_host_new
262func cs_pos_tokens(qb: *u8, qsz: i64, outs: *i64) -> i64
called by 1: main calls 2: ss_r32ss_vr
294func sc_conf_val(cf: *u8, n: i64, key: *u8, out: *u8, cap: i64) -> i64
called by 2: sc_conf_intmain calls 1: sc_lit_at
314func sc_conf_int(cf: *u8, n: i64, key: *u8) -> i64
called by 1: main calls 2: sc_conf_valsc_atoi_at
321func main(argc: i64, argv: *i64) -> i64