code wiki / _hdl_build / nx_ppmi_svd.nx

nx_ppmi_svd.nx

buildroot/runtime/_hdl_build/nx_ppmi_svd.nx

3327 B70 linesdepth 4pulls 4 transitivereach 1 importersview sourcekind librarytopic ppmi
docsdependenciesstructsconstsfunctions

about

nx_ppmi_svd.nx -- the sovereign semantic upgrade (Researcher-specced, built bits-up): PPMI weighting over the team's co-occurrence counts. Raw co-occurrence over-credits FREQUENT terms (a term that is in every doc co-occurs with everything, so it inflates unrelated docs). Pointwise Mutual Information fixes this: PMI(a,b) = log( P(a,b) / (P(a)P(b)) ) = log( cooc(a,b)*D / (df(a)*df(b)) ); PPMI keeps the POSITIVE part, so a term that co-occurs only AT CHANCE scores 0 and stops polluting retrieval. This is the count-based dense-embedding rung (Levy & Goldberg 2014: PPMI is competitive with word2vec; the further SVD dimensionality reduction is the next sub-rung, flagged not faked). Integer-only: PMI uses the fixed-point ln from nx_bm25. license_tier: ORIGINAL

dependencies 3 imports · 1 importers

nx_research_extract.nx nx_bm25.nx nx_syscalls.nx nx_ppmi_svd.nx nx_ppmi_svd_test.nx

imports: nx_research_extract.nxnx_bm25.nxnx_syscalls.nx

imported by: nx_ppmi_svd_test.nx

structs

none

consts

none

functions

14func ps_doc(ptrs: *i64, k: i64) -> *u8 { return ptrs[k] as *u8 }
17func ps_df(ptrs: *i64, lens: *i64, N: i64, term: *u8) -> i64
called by 1: ps_build_qvec calls 2: re_hasps_doc
22func ps_cooc(ptrs: *i64, lens: *i64, N: i64, a: *u8, b: *u8) -> i64
29func ps_ppmi_micro(cooc: i64, df_a: i64, df_b: i64, D: i64) -> i64
called by 1: ps_build_qvec calls 1: bm_ln_micro
39func ps_build_qvec(ptrs: *i64, lens: *i64, N: i64, qterm: *u8, vocab: *i64, V: i64, D: i64, qvec: *i64) -> i64
called by 1: main calls 3: ps_dfps_coocps_ppmi_micro
53func ps_score(ptrs: *i64, lens: *i64, k: i64, qvec: *i64, vocab: *i64, V: i64) -> i64
called by 2: ps_bestmain calls 2: re_hasps_doc
59func ps_best(ptrs: *i64, lens: *i64, N: i64, qvec: *i64, vocab: *i64, V: i64) -> i64
called by 1: main calls 1: ps_score
66func ps_build_raw_qvec(ptrs: *i64, lens: *i64, N: i64, qterm: *u8, vocab: *i64, V: i64, qvec: *i64) -> i64
called by 1: main calls 1: ps_cooc