code wiki / _hdl_build / nx_qabench_engine.nx

nx_qabench_engine.nx

buildroot/runtime/_hdl_build/nx_qabench_engine.nx

47933 B1278 linesdepth 3pulls 4 transitivereach 22 importersview sourcekind benchtopic qabench
docsdependenciesstructsconstsfunctions

about

nx_drbench.nx -- R2: the FIRST public-benchmark run of the Nishi researcher (census axis [BEN]: "MEASURED scores on public benchmarks -- NEVER run"). Dataset = HotpotQA distractor validation rows 0..99 (banked by nx_drbench_fetch -> knowledge/fetched/drb_hotpot_p0.raw, sovereign TLS). Protocol = the published distractor eval: each question ships 10 paragraphs (2 gold + 8 distractors); systems retrieve + extract an answer; scored by SQuAD word-F1/EM (nx_qa_score_lib, integer permille) + gold-paragraph recall at 2. TWO MODES measured side by side: mode0 SINGLE-HOP lexical baseline: question-term overlap retrieval -> span heuristic extraction. mode1 MULTI-HOP (the DeepResearcher loop shape, mechanical): hop-1 best para/sentence -> harvest BRIDGE entities (cap-runs + hop-1 title terms not in the question) -> expanded second retrieval -> extract. The LOW numbers ARE the honest baseline (no LLM synth yet -- that is census rung R1). Teeth: T1 parse integrity (100 rows, 10 paras, nonempty q/a) T2 gold titles parsed + resolvable to paragraphs T3 NEG-CONTROL: predictions scored against the WRONG (rotated) golds must collapse to ~0 (metric not rigged) T4 non-degenerate retrieval floor (recall at 2 must beat the ~200 permille random floor) T5 report mode0 vs mode1 delta (measurement, not a forced win -- honest either way) NOTE: all state lives in ONE ctx table g: *i64 passed to every fn (the proven rctx pattern) -- statics hit the known multi-static subscript-read compiler bug (filed; PHASE-A/B bisect proof in the session log). ctx slots: 0 raw 1 blen 2 boff 3 cur 4 q 5 a 6 ty 7 gtb 8 gto 9 gtl 10 ngt 11 ptb 12 pto 13 ptl 14 np 15 sb 16 so 17 sl 18 sp 19 ns 20 nb 21 no 22 nl 23 tko 24 tkl 25 tks 26 tkc 27 ntk 28 qnb 29 qto 30 qtl 31 nqt 32 qk 33 tnb 34 tto 35 ttl 36 tts 37 ttc 38 nttk 39 shit 40 thit 41 bhit 42 psc 43 xsc 44 btb 45 bto 46 btl 47 nbt 48 cnd 49 cno 50 cnl 51 cnt 52 ncn 53 scr 54 p0 55 p1 56 ga expect_exit: 0 license_tier: ORIGINAL

dependencies 2 imports · 9 importers

nx_qa_score_lib.nx nx_itoa_lib.nx nx_qabench_engine.nx nx_beir_eval.nx nx_dr_ppmi_build.nx nx_drbench.nx nx_ppmi_lib.nx nx_ppmi_probe.nx nx_qabench.nx nx_recall_dense.nx nx_semcorpus_build.nx nx_semppmi_build.nx

imports: nx_qa_score_lib.nxnx_itoa_lib.nx

imported by: nx_beir_eval.nxnx_dr_ppmi_build.nxnx_drbench.nxnx_ppmi_lib.nxnx_ppmi_probe.nxnx_qabench.nxnx_recall_dense.nxnx_semcorpus_build.nxnx_semppmi_build.nx

structs

none

consts

24const K_MAGIC_262144: i64 = 262144
25const K_MAGIC_260000: i64 = 260000
26const K_MAGIC_5381: i64 = 5381
27const K_MAGIC_77245: i64 = 77245
28const K_MAGIC_1073741789: i64 = 1073741789
29const K_MAGIC_1073741783: i64 = 1073741783
30const K_MAGIC_15000: i64 = 15000
31const K_MAGIC_2000: i64 = 2000
32const K_MAGIC_2048: i64 = 2048
33const K_MAGIC_4096: i64 = 4096
34const K_MAGIC_1000000: i64 = 1000000

functions

37func db_w(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(1,s,n); return 0 }
42func db_n(v: i64) -> i64 { nxi_out(v); return 0 }
48func db_read_raw(g: *i64, path: *u8, cap: i64) -> i64
74func db_body_start(g: *i64, total: i64) -> i64
84func db_b(g: *i64, i: i64) -> i64
91func db_match_key(g: *i64, pos: i64, key: *u8) -> i64
called by 1: db_find_key calls 1: db_b
107func db_find_key(g: *i64, key: *u8) -> i64
119func db_skip_ws(g: *i64) -> i64
134func db_dec_str(g: *i64, out: *u8, cap: i64) -> i64
164func db_parse_str_arr(g: *i64, arena: *u8, acap: i64, offs: *i64, lens: *i64, maxn: i64) -> i64
187func db_parse_sentences(g: *i64) -> i64
232func db_semhash(buf: *u8, off: i64, len: i64) -> i64
246func db_bsearch_i64(a: *i64, n: i64, v: i64) -> i64
258func db_tok_is(buf: *u8, off: i64, len: i64, lit: *u8) -> i64
267func db_is_stop(buf: *u8, off: i64, len: i64) -> i64
326func db_in_question(g: *i64, buf: *u8, off: i64, len: i64, contentonly: i64) -> i64
344func db_norm_row(g: *i64) -> i64
417func db_sent_hits(g: *i64, s: i64) -> i64
called by 1: db_score_row calls 1: qs_tok_eq2
447func db_title_hits(g: *i64, p: i64) -> i64
calls 1: qs_tok_eq2
477func db_body_hits(g: *i64, p: i64) -> i64
calls 1: qs_tok_eq2
515func db_term_in_body(g: *i64, p: i64, k: i64) -> i64
called by 1: db_score_row calls 1: qs_tok_eq2
540func db_term_in_title(g: *i64, p: i64, k: i64) -> i64
called by 1: db_score_row calls 1: qs_tok_eq2
560func db_score_row(g: *i64) -> i64
612func db_bridge_title_hits(g: *i64, p: i64) -> i64
called by 2: mainqab_run calls 1: qs_tok_eq2
639func db_bridge_body_hits(g: *i64, p: i64) -> i64
called by 2: mainqab_run calls 1: qs_tok_eq2
675func db_candidates(g: *i64, soff: i64, slen: i64) -> i64
771func db_cand_is_echo(g: *i64, ci: i64) -> i64
789func db_prefer_numeric(g: *i64) -> i64
804func db_wants_year(g: *i64) -> i64
called by 2: rd_featdb_sweep_numeric calls 1: db_tok_is
818func db_cand_is_common(g: *i64, ci: i64) -> i64
850func db_cand_multiword(g: *i64, ci: i64) -> i64
860func db_sweep_numeric(g: *i64, pa: i64, pb: i64, pred: *u8) -> i64
915func db_wants_location(g: *i64) -> i64
called by 1: qa_read_source calls 1: db_tok_is
926func db_sentence_has_locative(g: *i64, s: i64) -> i64
called by 1: db_sweep_location calls 1: db_tok_is
950func db_sentence_has_comparison(g: *i64, s: i64) -> i64
called by 1: db_sweep_location calls 1: db_tok_is
972func db_pick_proper_in(g: *i64, s: i64, pred: *u8) -> i64
991func db_sweep_location(g: *i64, pa: i64, pb: i64, pred: *u8) -> i64
1019func db_is_yesno(g: *i64) -> i64
1034func db_setpred(dst: *u8, src: *u8, len: i64) -> i64
1044func db_extract(g: *i64, s: i64, pred: *u8) -> i64
1103func db_bridge_from(g: *i64, s: i64, p: i64) -> i64
1173func db_best_sentence(g: *i64) -> i64
1189func db_best_sentence_in(g: *i64, p: i64) -> i64
called by 2: mainqab_run calls 1: qs_tok_eq2
1229func db_recall2(g: *i64, pa: i64, pb: i64) -> i64
called by 1: main
1264func db_top_para(g: *i64, skip: i64) -> i64