code wiki / _hdl_build / nx_library_search.nx

nx_library_search.nx

buildroot/runtime/_hdl_build/nx_library_search.nx

4889 B94 linesdepth 4pulls 4 transitivereach 5 importersview sourcekind librarytopic library
docsdependenciesstructsconstsfunctions

about

nx_library_search.nx -- S-class search/retrieval over a LIBRARY of information (not one source). The Researcher recognizes an index page as a LIBRARY, harvests every doc it lists, then answers a SEARCH PROMPT by RANKING the whole corpus -- TF-IDF style, which exceeds naive substring search: * substring search can only say present/absent in one blob and cannot rank or pick the best doc * this ranks N docs by sum over query terms of tf(term,doc) * idf(term), where RARER terms (low document-frequency) weigh MORE -- so a doc that matches the distinctive query terms wins over a doc that merely mentions a common one. That is the retrieval quality real search engines use. Corpus is held as parallel arrays: ptrs[k] = (doc text *u8 stored as i64), lens[k] = its length. license_tier: ORIGINAL Pairs with nx_research_extract (re_count/re_has/re_find) + nx_library_cache.

dependencies 3 imports · 5 importers

nx_research_extract.nx nx_bm25.nx nx_syscalls.nx nx_library_search.nx nx_engineer_wire_sclass_gate.nx nx_engineer_wire_test.nx nx_library_search_gate.nx nx_library_search_test.nx nx_referee_test.nx

imports: nx_research_extract.nxnx_bm25.nxnx_syscalls.nx

imported by: nx_engineer_wire_sclass_gate.nxnx_engineer_wire_test.nxnx_library_search_gate.nxnx_library_search_test.nxnx_referee_test.nx

structs

none

consts

none

functions

18func ls_is_library(text: *u8, n: i64) -> i64 { if re_count(text, n, "[doc] " as *u8) >= 3 { return 1 } return 0 }
called by 1: main calls 1: re_count
19func ls_entry_count(text: *u8, n: i64) -> i64 { return re_count(text, n, "[doc] " as *u8) }
called by 1: main calls 1: re_count
23func ls_entry_path(text: *u8, n: i64, k: i64, out: *u8, cap: i64) -> i64
called by 1: main calls 1: re_find
46func ls_doc(ptrs: *i64, k: i64) -> *u8 { return ptrs[k] as *u8 }
called by 2: ls_tfls_df
49func ls_tf(ptrs: *i64, lens: *i64, k: i64, term: *u8) -> i64 { return re_count(ls_doc(ptrs, k), lens[k], term) }
called by 1: ls_score calls 2: re_countls_doc
52func ls_df(ptrs: *i64, lens: *i64, ndocs: i64, term: *u8) -> i64
called by 3: ls_scoremainmain calls 2: re_hasls_doc
59func ls_idf(ndocs: i64, df: i64) -> i64 { return ndocs - df + 1 }
called by 1: ls_score
62func ls_score(ptrs: *i64, lens: *i64, ndocs: i64, k: i64, qterms: *i64, nq: i64) -> i64
called by 1: ls_best calls 3: ls_tfls_idfls_df
74func ls_best(ptrs: *i64, lens: *i64, ndocs: i64, qterms: *i64, nq: i64) -> i64
called by 4: mainmainmainmain calls 1: ls_score
86func ls_best_bm25(ptrs: *i64, lens: *i64, ndocs: i64, qterms: *i64, nq: i64) -> i64