code wiki / _hdl_build / nx_search_census.nx
nx_search_census.nx
buildroot/runtime/_hdl_build/nx_search_census.nx
about
nx_search_census.nx -- G1: THE SEARCH CORPUS, THROUGHPUT AND DIVERSITY CENSUS.
/compare/search rung G1, contracts idx_corpus_census and wc_throughput_census.
WHY IT EXISTS. On 2026-09-03 the operator could see that the index was not growing and the BOARD COULD
NOT. /compare/search carried four latency rows and a relevance row and NOTHING for corpus size, crawl
throughput or host diversity; the one row that touched scale had already withheld its Nishi figure as
"prose, never a measurement" and nobody had measured it since. Nothing else in the estate measured it
either -- knowledge/status/growth_series.log tracks the PM board, debts and maturity and carries no
corpus field at all -- so the one axis the operator was complaining about was invisible to every
consumer and therefore priced at zero by the ranker.
WHAT IT REFUSES TO DO. It prints LEVELS and never a rate. A rate needs two samples in time and the
estate ALREADY has a time spine, nx_growth_series, whose report verb computes d/w/m deltas per axis. A
second spine here would be the duplicate-ruler defect, so this organ is a PRODUCER for that one: it
TRUNCATE-writes a canonical status file and lets the spine carry the trajectory. The throughput axis
is the DIFFERENCE between two spine rows, never a number this organ invents.
EVERY COUNT CARRIES ITS COVERAGE, and that is not a slogan here. The key census is COMPLETE over the
manifest and says so, with its partition summed and the sum printed. The outcome partition is a
DECLARED TAIL WINDOW and says so. This organ was written in the same session a sibling validator was
caught reading the OLDEST 3.2 percent of a 129 MB log while printing "sampling newest-first" -- a
window published without its denominator reads as a population, and that is exactly how a three-week
old fossil became evidence about today.
ONE FAIRNESS RULER. The fair verdict is wc_frontier_fair, imported from the ENFORCER itself, and the
host parser is the crawler's own wc_host. Re-deriving either here would be a second ruler that agrees
today and diverges silently on the next edit to one side.
DECLARED GAPS, NAMED RATHER THAN HIDDEN. (1) doc= counts seg-store ENTRIES, which are append-only, so
it is an UPPER BOUND on distinct documents and the output says so on its own line; distinct-key dedup
is the named next rung. (2) The fr: pending-versus-done split is NOT reported: splitting it needs an
ss_hget per frontier key and the live shard holds millions, so the honest choices were a silent sample
or a named absence, and this takes the absence.
usage: nx_search_census [shard-prefix] [outcomes-log]
license_tier: ORIGINAL expect_exit: 0
dependencies 2 imports · 1 importers
imports: nx_web_crawl_step.nxnx_intlog.nx
imported by: nx_search_census_gate.nx
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 39 | const SC_SHARD: *u8 = "knowledge/store/dp-web-pub-" |
| 40 | const SC_LOG: *u8 = "knowledge/status/crawl_outcomes.log" |
| 41 | const SC_STATUS: *u8 = "knowledge/status/search_census.status" |
| 44 | const SC_LOGCAP: i64 = 4194304 |
| 49 | const SC_HOSTSLOTS: i64 = 65536 |
| 50 | const SC_HOSTBUF: i64 = 64 |
| 51 | const SC_URLBUF: i64 = 4096 |
| 52 | const SC_STATBUF: i64 = 4096 |
| 53 | const SC_HASH_MUL: i64 = 1000003 |
| 54 | const SC_TAB: i64 = 9 |
| 55 | const SC_NL: i64 = 10 |
| 56 | const SC_PERMIL: i64 = 1000 |
| 249 | const SC_CROSSOVER_CONF: *u8 = "knowledge/search_crossover.conf" |
| 250 | const SC_POS_EXT: *u8 = ".pos" |
| 251 | const SC_POS_MAGIC_N: i64 = 78 |
| 252 | const SC_POS_MAGIC_X: i64 = 88 |
| 253 | const SC_POS_MAGIC_Q: i64 = 81 |
| 254 | const SC_POS_MAGIC_1: i64 = 49 |
| 255 | const SC_POS_HDR: i64 = 8 |
| 256 | const SC_CONF_VALCAP: i64 = 2048 |
| 257 | const SC_INTBUF: i64 = 64 |
| 258 | const SC_SPACE: i64 = 32 |
| 259 | const SC_CR: i64 = 13 |
functions
| 58 | func sc_w(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 } called by 1: main |
| 59 | func sc_n(v: i64) -> i64 called by 1: main |
| 67 | func sc_cat(out: *u8, o: i64, s: *u8) -> i64 { var i: i64 = 0; while s[i] != (0 as u8) { out[o] = s[i]; o = o + 1; i = i + 1 } return o } |
| 68 | func sc_catn(out: *u8, o: i64, v: i64) -> i64 called by 1: main |
| 79 | func sc_lit_at(p: *u8, len: i64, lit: *u8) -> i64 |
| 89 | func sc_key_at(kb: *u8, eo: i64, kl: i64, lit: *u8) -> i64 |
| 97 | func sc_atoi_at(p: *u8, s: i64, e: i64) -> i64 |
| 103 | func sc_hash(p: *u8, n: i64) -> i64 called by 1: sc_host_new |
| 111 | func sc_host_new(tbl: *i64, slots: i64, hp: *u8, hl: i64) -> i64 |
| 134 | func idx_corpus_census(h: *i64, out: *i64) -> i64 |
| 176 | func wc_throughput_census(buf: *u8, n: i64, out: *i64, hosttbl: *i64, slots: i64, hbuf: *u8, ubuf: *u8) -> i64 |
| 262 | func cs_pos_tokens(qb: *u8, qsz: i64, outs: *i64) -> i64 |
| 294 | func sc_conf_val(cf: *u8, n: i64, key: *u8, out: *u8, cap: i64) -> i64 |
| 314 | func sc_conf_int(cf: *u8, n: i64, key: *u8) -> i64 |
| 321 | func main(argc: i64, argv: *i64) -> i64 |