code wiki / _hdl_build / nx_semcorpus_build.nx
nx_semcorpus_build.nx
buildroot/runtime/_hdl_build/nx_semcorpus_build.nx
about
nx_semcorpus_build.nx -- R3b-scale CORPUS-LEVEL distributional embeddings (the rung after 3 measured-negative
per-question climbs; see nx_qabench MODE2 header). Builds ONE persisted hashed count-embedding matrix from
ALL banked benchmark context prose (~4.5MB Wikipedia-register: qab_squad_big + qab_hotpot_big +
qab_musique_big), then nx_qabench's MODE2 loads it and selects sentences by corpus-informed cosine.
METHOD (sovereign, integer, no training loop -- Random-Indexing/feature-hashing lineage): text = every JSON
string value >= 48 chars (paragraphs+questions; SHORT strings -- keys, titles, GOLD ANSWERS -- are
structurally excluded = no label leakage; corpus is the inference-visible context pool, unsupervised).
token row = djb2(token) % 16384; for each content token, each stop-filtered context token within +-4 hashes
to one of 256 dims: M[row*256 + dim] += 1. Persisted -> knowledge/index/semcorpus_v1.bin (NXSEMC1 header).
HONEST CAVEATS baked in: hash collisions (~2-4 words/row) + raw counts (no PPMI) = v1; probes below REPORT
whether corpus stats order won~defeated > won~purple (the exact wall) -- reported, not asserted.
TEETH: T1 strings>=3000 T2 token-events>=200000 T3 nonzero cells>=50000 T4 write+reload spot-check
T5 NEG: identity cos=1000 and a gibberish token's empty row scores 0 vs everything.
expect_exit: 0 license_tier: ORIGINAL
dependencies 1 imports · 0 importers
imports: nx_qabench_engine.nx
imported by: nobody (leaf or entry point)
call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown
structs
| none |
consts
| 16 | const SC_MAGIC_5381: i64 = 5381 |
| 17 | const SC_MAGIC_1073741789: i64 = 1073741789 |
| 18 | const SC_MAGIC_16777216: i64 = 16777216 |
| 19 | const SC_MAGIC_250000: i64 = 250000 |
| 20 | const SC_MAGIC_4000: i64 = 4000 |
| 21 | const SC_MAGIC_2654435761: i64 = 2654435761 |
| 22 | const SC_MAGIC_262144: i64 = 262144 |
| 23 | const SC_MAGIC_32768: i64 = 32768 |
| 24 | const SC_MAGIC_4096: i64 = 4096 |
| 25 | const SC_MAGIC_65536: i64 = 65536 |
| 26 | const SC_MAGIC_3000: i64 = 3000 |
| 27 | const SC_MAGIC_200000: i64 = 200000 |
| 28 | const SC_MAGIC_50000: i64 = 50000 |
| 30 | const SC_ROWS: i64 = 16384 |
| 31 | const SC_DIMS: i64 = 256 |
| 32 | const SC_WIN: i64 = 4 |
| 33 | const SC_MINSTR: i64 = 48 |
functions
| 35 | func sc_hash(buf: *u8, off: i64, len: i64) -> i64 |
| 42 | func sc_isqrt(v: i64) -> i64 |
| 51 | func sc_cos_rows(M: *i64, ra: i64, rb: i64) -> i64 |
| 74 | func sc_probe(M: *i64, a: *u8, b: *u8) -> i64 |
| 79 | func sc_len(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } return n } called by 1: sc_probe |
| 83 | func sc_ingest(g: *i64, path: *u8, M: *i64, c: *i64) -> i64 |
| 137 | func main() -> i64 |