code wiki / _hdl_build / nx_research_grow.nx

nx_research_grow.nx

buildroot/runtime/_hdl_build/nx_research_grow.nx

16427 B318 linesdepth 5pulls 8 transitivereach 0 importersview sourcekind tooltopic research
docsdependenciesstructsconstsfunctions

about

nx_research_grow.nx -- THE SELF-GROWING RESEARCHER INDEX (R4 self-grow loop) WITH CONTENT-DEDUP + CROSS-TOPIC FLAGGING. Each run indexes the DELTA (fetched docs not yet SEEN) into a NEW append-only SHARD, but FIRST fingerprints each doc's CONTENT (FNV-1a 64-bit). If that content fingerprint is already registered, the doc is NOT re-indexed (no duplicate as coverage grows) -- instead the registry records the new (topic,path) for that fingerprint, and if the new topic differs from the canonical topic we EMIT A CROSS-TOPIC FLAG: "this information serves two research topics, found in the same place." So the corpus compounds on every use, never duplicates content, never loses data, and the structure intelligently links topics that share sources. Bounded batch per run = OOM-safe (no sys_munmap). Reuses the FIXED nx_inv build + nx_dirent walk. Topic = the filename prefix before the first '_' (perma_/sem_/food_/ng_/...). expect_exit: 0 license_tier: ORIGINAL

dependencies 3 imports · 0 importers

nx_search_inverted_persist.nx nx_itoa_lib.nx nx_dirent.nx nx_research_grow.nx

imports: nx_search_inverted_persist.nxnx_itoa_lib.nxnx_dirent.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main rg_puts sys_mkdir sys_read_file sys_openat_rd sys_lseek sys_mmap sys_read sys_close rg_field_end rg_parse_dec rg_catbuf rg_cat rg_catnum sys_openat_wr sys_openat_append nx_open_rd nx_openat nx_dirent_read nx_dirent_iter nx_dirent_name_len rg_indexable rg_seen rg_topic rg_fp rg_streq rg_emit rg_num nxi_out nxi_fd sys_mmap ↻ ccz_cat_num sys_write sys_munmap nx_inv_build_from_jsonl sys_read_file ↻ nx_inv_new_sized nx_inv_index_row nx_inv_is_token_char nx_inv_hash_bytes_lower

structs

none

consts

14const RG_DIR_BUF: i64 = 65536
15const RG_PATH_CAP: i64 = 4096
16const RG_BATCH_MAX: i64 = 50 // bounded files-read per run -> OOM-safe; corpus compounds across runs
17const RG_REG_CAP: i64 = 65536 // max distinct content fingerprints tracked in memory
18const RG_TOP_BUF: i64 = 4194304 // canonical-topic string arena

functions

20func rg_puts(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(1,s,n); return 0 }
called by 1: main
25func rg_num(v: i64) -> i64 { nxi_out(v); return 0 }
called by 1: main calls 1: nxi_out
26func rg_cat(dst: *u8, off: i64, s: *u8) -> i64 { var i: i64=0; while s[i]!=(0 as u8){dst[off+i]=s[i];i=i+1} return off+i }
called by 1: main
27func rg_catbuf(dst: *u8, off: i64, src: *u8, n: i64) -> i64 { var i: i64=0; while i<n { dst[off+i]=src[i]; i=i+1 } return off+n }
called by 1: main
28func rg_catnum(dst: *u8, off: i64, v: i64) -> i64
called by 1: main
37func rg_indexable(name: *u8, n: i64) -> i64
called by 1: main
45func rg_topic(name: *u8, nlen: i64, out: *u8) -> i64
called by 1: main
57func rg_streq(a: *u8, alen: i64, b: *u8, blen: i64) -> i64
called by 1: main
65func rg_seen(man: *u8, mlen: i64, path: *u8, plen: i64) -> i64
called by 1: main
82func rg_field_end(buf: *u8, i: i64, end: i64) -> i64
called by 1: main
93func rg_parse_dec(buf: *u8, start: i64, fend: i64) -> i64
called by 1: main
100func rg_emit(corpus_fd: i64, buf: *u8, len: i64) -> i64
called by 1: main
121func rg_fp(buf: *u8, len: i64) -> i64
called by 1: main
134func main() -> i64