code wiki / _hdl_build / nx_bm25f.nx
nx_bm25f.nx
buildroot/runtime/_hdl_build/nx_bm25f.nx
about
nx_bm25f.nx -- BM25F: fielded BM25 (Robertson/Zaragoza "Simple BM25 extension to multiple weighted fields",
CIKM'04 -- cited sovereign-fetched lib_bm25.txt: "BM25F defines each type of field as a stream, applying a
per-stream weighting"). The honest relevance rung above plain BM25: a query term in the TITLE field should
outrank the same term buried in the BODY. The fix is exactly what nx_bm25's own header predicted -- "a thin
extension via a weight table". Two fields here (title, body); adding more (url/anchor/tags) = more rows in
the weight table, no formula change (Cardinal 11/22).
SIMPLE BM25F (the correct form -- length-normalize + weight PER FIELD, then saturate ONCE over the combined
pseudo-frequency, so a title hit isn't double-counted):
pseudo_tf(t,d) = SUM_f w_f * tf(t,f,d) / ( 1 - b + b * dl_f/avgdl_f )
score(d,q) = SUM_t IDF(t) * pseudo_tf*(k1+1) / ( k1 + pseudo_tf )
Integer/fixed-point throughout (reuses nx_bm25's bm_idf_micro keystone), x1e6 scale like bm_score.
license_tier: ORIGINAL
dependencies 1 imports · 3 importers
imports: nx_bm25.nx
imported by: nx_bm25f_gate.nxnx_onsite_search.nxnx_wiki_answer.nx
structs
| none |
consts
| 16 | const BMF_K1_X1000: i64 = 1200 // k1 = 1.2 (tf saturation) |
| 17 | const BMF_B_X1000: i64 = 750 // b = 0.75 (length normalization) |
| 18 | const BMF_W_TITLE: i64 = 3 // title-field boost inside the per-field pseudo-tf (weight table) |
| 19 | const BMF_W_BODY: i64 = 1 // body-field boost |
| 26 | const BMF_TITLE_MATCH_MULT: i64 = 2 // each title-matched query term adds (BMF_TITLE_MATCH_MULT * IDF) to score |
functions
| 29 | func bmf_contrib(w: i64, tf: i64, dl: i64, avgdl: i64) -> i64 called by 1: bmf_score |
| 39 | func bmf_df(titles: *i64, tlens: *i64, bodies: *i64, blens: *i64, N: i64, term: *u8) -> i64 |
| 52 | func bmf_score(titles: *i64, tlens: *i64, dlt: *i64, |
| 80 | func bmf_best(titles: *i64, tlens: *i64, dlt: *i64, |