code wiki / (root) / nx_seg_store.nx

nx_seg_store.nx

buildroot/runtime/nx_seg_store.nx

143853 B3231 linesdepth 2pulls 2 transitivereach 1272 importersview sourcekind librarytopic seg
docsdependenciesstructsconstsfunctions

about

nx_seg_store.nx -- PART A of the sovereign storage substrate rung 1 (knowledge/specs/2026-06-09-tutoring-storage-substrate-rung1.md). Append-only IMMUTABLE segment store, pure NishiLang, NO SQL. Model: - a segment is written once (seg-<id>.docs) and never mutated - new data => a NEW segment; deletes => tombstone entries (ADDITIVE law: history is never destroyed, old versions stay readable) - THE COMMIT POINT: segment temp file -> rename(2), then manifest temp -> rename(2). Readers only see segments listed in manifest.txt, so a half-written segment (crash before commit) is INVISIBLE by construction. That is the crash-safety claim the gate proves by fault injection. Entry format inside .docs: u8 kind (1=put 2=tombstone) | u32be klen | key | u32be vlen | value-bytes Values are nx_canon_cid canonical record bytes (content-addressed upstream). "Directory" = a path PREFIX string (e.g. "/tmp/im123-"), so no mkdir is needed (sys_mkdir landmine avoided); a durable store uses a knowledge/ prefix. Rung-2 additions 2026-06-10: sys_fsync (probe-proven x86 74) on every file write + directory fsync after the commit renames = the IM3 power-loss debt CLOSED at the code level; per-segment sorted key index seg-<id>.keys with binary-search reads (ss_get_idx), scan path kept as the ORACLE the gate races it against. Honest remaining scope (flagged, NOT silent caps): term-postings for text search (IM2 full: delta+bitpack), manifest cap 256 segments + 4096 keys/segment (compaction/merge = IM4). license_tier: ORIGINAL

dependencies 1 imports · 416 importers

nx_syscalls.nx nx_seg_store.nx _gallery_serve_ref.nx _galx_bulk_ingest.nx _galx_ingest_gate.nx _galx_prod_ingest.nx _galx_serve_setup.nx _ixprof.nx _queue_load_gate_authored.nx _ss_compact_cap_gate.nx _wiki_archive_gate.nx _wiki_search_gate.nx

diagram shows first 10 each side; +0 more imports, +406 more importers in the complete lists below.

imports: nx_syscalls.nx

imported by: _gallery_serve_ref.nx_galx_bulk_ingest.nx_galx_ingest_gate.nx_galx_prod_ingest.nx_galx_serve_setup.nx_ixprof.nx_queue_load_gate_authored.nx_ss_compact_cap_gate.nx_wiki_archive_gate.nx_wiki_search_gate.nxnx_account_admin_exceed_gate.nxnx_acct_admin_handoff.nxnx_activities.nxnx_activities_gate.nxnx_ad_feedback_gate.nxnx_ad_intake.nxnx_ad_ivt_gate.nxnx_ad_store.nxnx_ad_store_seed.nxnx_ads.nxnx_adversarial_sov.nxnx_aid.nxnx_aid_federation.nxnx_aid_federation_gate.nxnx_aid_gate.nxnx_algo_registry.nxnx_algo_registry_migrate.nxnx_algo_retire.nxnx_analyst_store.nxnx_app_store.nxnx_arch_board.nxnx_archcensus.nxnx_ask_lib.nxnx_asset_catalog.nxnx_asset_catalog_gate.nxnx_asset_record.nxnx_asset_record_gate.nxnx_asset_signals.nxnx_asset_signals_gate.nxnx_asset_status_gate.nxnx_atlas_discover.nxnx_atlas_discover_gate.nxnx_atlas_gapscan.nxnx_atlas_gapscan_gate.nxnx_audio_census.nxnx_autofix_cascade_gate.nxnx_autofix_ensemble.nxnx_backlog_feeder.nxnx_battery_grade.nxnx_bench_ingest.nxnx_bills_store.nxnx_blocklist_store.nxnx_browser_census.nxnx_browser_profile_store.nxnx_build_reg_sov.nxnx_cap_gap_sov.nxnx_cap_harvest.nxnx_cap_provision_gate.nxnx_capability_no_regression.nxnx_carflip_deal.nxnx_census_load.nxnx_chart_migrate.nxnx_ciaaw_gate.nxnx_ciaaw_ingest.nxnx_ciq_analyst.nxnx_ciq_field_clinical.nxnx_ciq_field_connect.nxnx_ciq_field_games.nxnx_ciq_field_imagegen.nxnx_ciq_field_maintenance.nxnx_ciq_field_omniforge.nxnx_ciq_publish.nxnx_ciq_quadrant.nxnx_ciq_sclass_gate.nxnx_ciq_visual.nxnx_clawf_route.nxnx_clean_market.nxnx_clean_market_gate.nxnx_climbout_progress.nxnx_coach_curriculum.nxnx_coach_exceed.nxnx_coach_rubric.nxnx_coach_serve.nxnx_codata_gate.nxnx_codata_ingest.nxnx_collision_watchdog_gate.nxnx_common_tasks.nxnx_companion_gate.nxnx_conductor_recall_gate.nxnx_conductor_registry.nxnx_consol_apply.nxnx_consol_apply_gate.nxnx_consol_apply_mc.nxnx_consol_apply_mc_gate.nxnx_consolidate.nxnx_consolidate_gate.nxnx_contour_print.nxnx_contour_print_gate.nxnx_council_gate.nxnx_crm_store.nx +316 more (shown cap 100 declared)

structs

none

consts

29const SS_MAGIC_16777619: i64 = 16777619
30const SS_MAGIC_65536: i64 = 65536
31const SS_MAGIC_1048576: i64 = 1048576
32const SS_MAGIC_4096: i64 = 4096
33const SS_MAGIC_1024: i64 = 1024
34const SS_MAGIC_1039: i64 = 1039
35const SS_MAGIC_1040: i64 = 1040
36const SS_MAGIC_1071: i64 = 1071
37const SS_MAGIC_1072: i64 = 1072
38const SS_MAGIC_1279: i64 = 1279
39const SS_MAGIC_12352: i64 = 12352
40const SS_MAGIC_12543: i64 = 12543
41const SS_MAGIC_3040: i64 = 3040
42const SS_MAGIC_13312: i64 = 13312
43const SS_MAGIC_19903: i64 = 19903
44const SS_MAGIC_3400: i64 = 3400
45const SS_MAGIC_19968: i64 = 19968
46const SS_MAGIC_40959: i64 = 40959
47const SS_MAGIC_44032: i64 = 44032
48const SS_MAGIC_55215: i64 = 55215
49const SS_MAGIC_63744: i64 = 63744
50const SS_MAGIC_64255: i64 = 64255
51const SS_MAGIC_1469598103934665603: i64 = 1469598103934665603
52const SS_MAGIC_1099511628211: i64 = 1099511628211
932const SS_IMP_K: i64 = 512
1199const SS_SEGID_PTRBAND_LO: i64 = 10000000000000
1200const SS_SEGID_PTRBAND_HI: i64 = 140737488355328
1323const SS_MANIFEST_LEGACY_CAP: i64 = 256
1324const SS_VER_SLOTS: i64 = 260 // per-key version buffer allocation (kinds/ptrs/lens/srcs slots)
1325const SS_VER_WINDOW: i64 = 256 // versions of ONE key a scan retains; on overflow the OLDEST slides out
1380const SS_MF_MAGIC: i64 = 0x53534D46
1776const SSL_EMPTY: i64 = 0 - 1
2039const SSC_SLOTS: i64 = 16
2040const SSC_W: i64 = 4
2042const SSC_RETIRE: i64 = 32
2901const SS_CUR_SLOTS: i64 = 8

functions

54func ss_len(s: *u8) -> i64 {
called by 7: ss_addss_scan_seglistss_idx_findss_terms_findss_terms_ordinalss_compact+1
60func ss_cat(dst: *u8, off: i64, s: *u8) -> i64 {
called by 130: mainmainmainam_metapathmainad_selftest+124
67func ss_catn(dst: *u8, off: i64, v: i64) -> i64 {
called by 56: mainmainad_selftestmainag_mkpfxmain+50 calls 1: sys_mmap
80func ss_w32(p: *u8, off: i64, v: i64) -> i64 {
called by 8: ec_packss_add2ss_build_keysss_build_termsss_build_impss_write_seg+2
88func ss_r32(p: *u8, off: i64) -> i64 {
called by 137: gs_grid_bodycs_fieldcs_forecastcs_givingec_axisec_layer+131
97func ss_readall(path: *u8, szout: *i64) -> *u8 {
called by 158: xp_parseag_migratemainmainmaincm_migrate+152 calls 5: sys_openat_rdsys_lseeksys_mmapsys_readsys_close
122func ss_loadfile(path: *u8, szout: *i64, usemmap: i64) -> *u8 {
called by 7: dss_search_off_divsib_oness_load_aux2ss_open2ss_term_topmain+1 calls 2: sys_map_filess_readall
132func ss_writefile(path: *u8, bytes: *u8, n: i64) -> i64 {
called by 71: mainmainmainmainmainmain+65 calls 4: sys_openat_wrsys_writesys_closesys_fsync
150func ss_syncdir(prefix: *u8) -> i64 {
called by 5: ss_commitss_sync_nowss_compactss_compact_capmain calls 4: sys_mmapsys_openat_rdsys_fsyncsys_close
176func ss_begin_cap(cap: i64) -> *i64 {
called by 23: mainem_rememberdo_retypemvc_batch_add_pfxmw_flush_planemw_run+17 calls 1: sys_mmap
184func ss_begin() -> *i64 {
called by 266: mainmainbuild_storemainmainmain+260 calls 1: ss_begin_cap
188func ss_add(w: *i64, kind: i64, key: *u8, val: *u8, vlen: i64) -> i64 {
called by 290: mainbuild_storemainah_fileac_putmain+284 calls 2: ss_add2ss_len
193func ss_add2(w: *i64, kind: i64, key: *u8, kl: i64, val: *u8, vlen: i64) -> i64 {
called by 6: maindo_retypess_addss_compactss_compact_capmain calls 1: ss_w32
210func ss_segname(prefix: *u8, segid: i64, tmp: i64, out: *u8) -> i64 {
called by 4: mainmainss_crashwritess_write_seg calls 2: ss_catss_catn
222func ss_kcmp(a: *u8, al: i64, b: *u8, bl: i64) -> i64 {
called by 12: ss_build_keysss_build_termsss_idx_findss_terms_findss_terms_ordinalss_compact+6
241func ss_build_keys(w: *i64, kout: *u8) -> i64 {
called by 5: mainss_write_segmainmainmain calls 4: sys_mmapss_r32ss_kcmpss_w32
375func ss_crashwrite(prefix: *u8, w: *i64, segid: i64) -> i64 {
called by 1: ig_child_write calls 3: sys_mmapss_segnamess_writefile
393func ss_vw(p: *u8, off: i64, v: i64) -> i64 {
called by 2: ss_build_termsss_build_imp
407func ss_vr(p: *u8, pos: *i64) -> i64 {
called by 6: ss_build_impss_termss_term_andss_q_positionsss_term_topss_phrase
424func ss_tok_table(tbl: *u8) -> i64 {
called by 13: maindss_correctdss_search_off_divdss_servedsv_doc_servedss_api_suggest+7
443func ss_fold_cp(cp: i64) -> i64 {
called by 2: cw_scriptmassss_tok_next2
460func ss_u8cp(b: *u8, sz: i64, i: i64) -> i64 {
called by 2: cw_scriptmassss_tok_next2
485func ss_fold_word_cp(cp: i64) -> i64 {
called by 2: cw_scriptmassss_tok_next2
496func ss_is_cjk(cp: i64) -> i64 {
called by 2: cw_scriptmassss_tok_next2
510func ss_tok_next2(b: *u8, sz: i64, pos: *i64, tout: *u8, tbl: *u8) -> i64 {
called by 13: dss_tfdss_tf_alldss_prox_alldss_correctdss_search_off_divdsv_tokq+7 calls 4: ss_u8cpss_fold_cpss_fold_word_cpss_is_cjk
597func ss_tok_next(b: *u8, sz: i64, pos: *i64, tout: *u8) -> i64 {
calls 3: sys_mmapss_tok_tabless_tok_next2
604func ss_tok_has(b: *u8, sz: i64, term: *u8) -> i64 {
called by 1: main calls 3: sys_mmapss_tok_tabless_tok_next2
628func ss_fnv(s: *u8) -> i64 {
called by 2: ss_build_termsse_walk_latest
653func ss_build_terms(w: *i64, tblob: *u8, pblob: *u8, qblob: *u8, louts: *i64) -> i64 {
called by 2: mainss_write_seg calls 8: sys_mmapss_tok_tabless_r32ss_tok_next2ss_fnvss_kcmp+2
937func ss_imp_thresh(tfs: *i64, n: i64, k: i64, outs: *i64) -> i64 {
called by 1: ss_build_imp calls 1: sys_mmap
966func ss_build_imp(tb: *u8, tsz: i64, pb: *u8, psz: i64, qb: *u8, qsz: i64, wblob: *u8) -> i64 {
called by 2: sib_oness_write_seg calls 6: ss_r32sys_mmapss_w32ss_vrss_imp_threshss_vw
1039func ss_auxname(prefix: *u8, segid: i64, ext: *u8, tmp: i64, out: *u8) -> i64 {
called by 2: ss_write_segmain calls 2: ss_catss_catn
1059func ss_write_seg(prefix: *u8, w: *i64, segid: i64) -> i64
called by 6: ss_commitss_commit_deferredss_compactss_compact_capmainmain calls 13: ss_segid_oksys_mmapss_catss_catnsys_writess_segname+7
1137func ss_load_aux(prefix: *u8, segname: *u8, outs: *i64) -> i64 {
called by 2: ss_idx_lookupsc_stat calls 1: ss_load_aux2
1140func ss_load_aux2(prefix: *u8, segname: *u8, outs: *i64, usemmap: i64) -> i64 {
called by 3: sib_oness_load_auxss_open2 calls 5: sys_mmapss_catss_loadfiless_r32sys_munmap
1203func ss_segid_ok(v: i64) -> i64
called by 6: mainss_write_segss_commitss_commit_deferredss_next_segidmain
1209func ss_commit(prefix: *u8, w: *i64, segid: i64) -> i64
called by 284: mainmainbuild_storemainmainmain+278 calls 10: ss_segid_oksys_mmapss_catss_catnsys_writess_write_seg+4
1271func ss_commit_deferred(prefix: *u8, w: *i64, segid: i64) -> i64
called by 1: reg_put_x calls 9: ss_segid_oksys_mmapss_catss_catnsys_writess_write_seg+3
1314func ss_sync_now(prefix: *u8) -> i64
called by 1: reg_batch_sync calls 1: ss_syncdir
1332func ss_manifest_file(prefix: *u8, fname: *u8, segs: *i64) -> i64 {
called by 8: ci_runmainss_manifestti_runtp_runts_ingest+2 calls 3: sys_mmapss_catss_readall
1361func ss_manifest(prefix: *u8, segs: *i64) -> i64 {
called by 58: gs_grid_bodyg_segcountads_seg_nextmainei_next_segidfd_seg_next+52 calls 1: ss_manifest_file
1381func ss_manifest_file_dyn(prefix: *u8, fname: *u8, out_segs: *i64) -> i64 {
called by 2: ss_manifest_dynss_scan_all calls 4: sys_mmapss_catss_readallsys_munmap
1431func ss_manifest_free(segs: *i64, n: i64) -> i64 {
called by 1: ss_open2 calls 1: sys_munmap
1442func ss_manifest_dyn(prefix: *u8, out_segs: *i64) -> i64 {
called by 33: cs_forecastcs_givingmainmainmx_storespj_gather+27 calls 1: ss_manifest_file_dyn
1452func ss_next_segid(prefix: *u8) -> i64 {
called by 48: nx_app_store_putab_store_putak_commitas_seg_nextbl_put_pfxcg_seg_next+42 calls 4: sys_mmapss_catss_readallss_segid_ok
1490func ss_manifest_file_cap(prefix: *u8, fname: *u8, segs: *i64, cap: i64) -> i64 {
called by 3: mainmainss_manifest_cap calls 3: sys_mmapss_catss_readall
1518func ss_manifest_cap(prefix: *u8, segs: *i64, cap: i64) -> i64 {
called by 20: mainmainmaings_grid_bodymainmain+14 calls 1: ss_manifest_file_cap
1526func ss_scan_cap(prefix: *u8, key: *u8, kinds: *i64, ptrs: *i64, lens: *i64, cap: i64) -> i64 {
called by 2: ss_get_capmain calls 3: sys_mmapss_manifest_capss_scan_seglist
1536func ss_get_cap(prefix: *u8, key: *u8, ptrout: *i64, lenout: *i64, cap: i64) -> i64 {
called by 16: count_livecheck_valbr2_get_rawbr2_registerednx_mbox_fetchnx_mbox_exists+10 calls 2: sys_mmapss_scan_cap
1555func ss_scan_seglist(prefix: *u8, segs: *i64, ns: i64, key: *u8, kinds: *i64, ptrs: *i64, lens: *i64, srcs: *i64, srcval: i64, cnt0: i64, cap: i64) -> i64 {
called by 3: ss_scan_capss_scanss_scan_all calls 5: ss_lensys_mmapss_catss_readallss_r32
1610func ss_scan(prefix: *u8, key: *u8, kinds: *i64, ptrs: *i64, lens: *i64) -> i64 {
called by 7: asr_load_field_winmainnx_mbox_history_countmainmx_historyss_get+1 calls 3: sys_mmapss_manifest_dynss_scan_seglist
1624func ss_scan_all(prefix: *u8, key: *u8, kinds: *i64, ptrs: *i64, lens: *i64, srcs: *i64) -> i64 {
called by 6: ex_histcountmainhau_emitscan_countmainmain calls 4: sys_mmapss_manifest_file_dynss_scan_seglistss_manifest_dyn
1639func ss_idx_find(b: *u8, sz: i64, key: *u8, voffout: *i64, vlenout: *i64) -> i64 {
called by 2: ss_idx_lookupss_hget calls 3: ss_r32ss_lenss_kcmp
1666func ss_idx_lookup(prefix: *u8, segname: *u8, key: *u8, voffout: *i64, vlenout: *i64) -> i64 {
called by 1: ss_get_idx calls 3: sys_mmapss_load_auxss_idx_find
1702func ss_close(h: *i64) -> i64 {
called by 9: dss_open_maybe_cachedmw_selftestmainraci_is_accountableraci_rowraci_accountable_for+3 calls 1: sys_munmap
1762func ss_open(prefix: *u8) -> *i64 {
called by 154: qlg_latest_hasrev_existsah_hasag_migrateag_gateag_count+148 calls 1: ss_open2
1778func ssl_pow2(n: i64) -> i64 {
called by 1: ss_open2
1784func ssl_total_keys(h: *i64, ns: i64) -> i64 {
called by 1: ss_open2 calls 1: ss_r32
1796func ssl_key_eq(h: *i64, seg: i64, eo: i64, kb2: *u8, eo2: i64) -> i64 {
called by 2: ssl_buildssl_lookup calls 1: ss_r32
1809func ssl_hash_entry(kb: *u8, eo: i64) -> i64 {
called by 2: ssl_buildssl_lookup calls 1: ss_r32
1823func ssl_build(h: *i64, ns: i64, tbl_seg: *i64, tbl_eo: *i64, cap: i64) -> i64 {
called by 1: ss_open2 calls 3: ss_r32ssl_hash_entryssl_key_eq
1860func ssl_lookup(h: *i64, tbl_seg: *i64, tbl_eo: *i64, cap: i64, kb: *u8, eo: i64) -> i64 {
called by 1: ss_open2 calls 2: ssl_hash_entryssl_key_eq
1873func ss_open2(prefix: *u8, usemmap: i64) -> *i64 {
called by 7: dss_open_maybe_cachedmainpb_runss_openmainmain+1 calls 12: sys_mmapss_manifest_dynss_catss_loadfiless_load_aux2ssl_total_keys+6
2050func ssc_init() -> i64 {
called by 3: ss_cache_reapss_cache_retiredss_open_cached calls 1: sys_mmap
2076func ssc_retire_handle(h: i64) -> i64 {
called by 1: ss_open_cached
2089func ss_cache_reap() -> i64 {
called by 2: mainmain calls 2: ssc_initss_close
2105func ss_cache_retired() -> i64 {
called by 1: main calls 1: ssc_init
2109func ssc_streq(a: *u8, b: *u8) -> i64 {
called by 1: ss_open_cached
2120func ssc_sig_of(prefix: *u8, sig: *i64) -> i64 {
called by 1: ss_open_cached calls 2: ss_catsys_fstatat
2140func ss_open_cached(prefix: *u8) -> *i64 {
called by 12: cat_listcat_countar_get_by_ciddp_readdp_get_policyet_count_targets_store+6 calls 5: ssc_initssc_sig_ofssc_streqss_openssc_retire_handle
2201func ss_hget(h: *i64, key: *u8, ptrout: *i64, lenout: *i64) -> i64 {
called by 167: qlg_latest_hasrev_existsah_hasag_migrateag_gateag_count+161 calls 2: sys_mmapss_idx_find
2225func ss_terms_find(tb: *u8, tsz: i64, term: *u8, outs: *i64) -> i64 {
called by 3: ss_termss_term_andss_term_dcount calls 3: ss_r32ss_lenss_kcmp
2259func ss_term(h: *i64, term: *u8, kpout: *i64, klout: *i64, max: i64) -> i64 {
called by 11: dss_search_off_divmainterm_countmainmainmain+5 calls 3: ss_terms_findss_vrss_r32
2312func ss_term_and(h: *i64, terms: *i64, nterms: i64, kpout: *i64, klout: *i64, max: i64) -> i64 {
called by 4: mainmainmainmain calls 4: ss_termss_terms_findss_vrss_r32
2407func ss_term_dcount(h: *i64, term: *u8) -> i64 {
called by 3: maindss_correctdss_search_off_div calls 1: ss_terms_find
2428func ss_term_count(h: *i64, s: i64) -> i64 {
called by 3: dss_correctdss_suggestdss_stem_expand calls 1: ss_r32
2435func ss_term_at(h: *i64, s: i64, idx: i64, tp_out: *i64, tl_out: *i64, dc_out: *i64) -> i64 {
called by 3: dss_correctdss_suggestdss_stem_expand calls 1: ss_r32
2456func ss_doc_count(h: *i64) -> i64 {
called by 2: maindss_search_off_div calls 1: ss_r32
2485func ss_terms_ordinal(tb: *u8, tsz: i64, term: *u8, outs: *i64) -> i64 {
called by 2: ss_term_topss_phrase calls 3: ss_r32ss_lenss_kcmp
2513func ss_q_positions(qb: *u8, qsz: i64, ord: i64, rank: i64, posout: *i64, maxpos: i64) -> i64 {
called by 1: ss_phrase calls 3: ss_r32sys_mmapss_vr
2555func ss_term_top(prefix: *u8, h: *i64, term: *u8, kpout: *i64, klout: *i64, tfout: *i64, max: i64, satout: *i64) -> i64 {
called by 2: dss_search_off_divmain calls 7: sys_mmapss_manifest_dynss_catss_loadfiless_terms_ordinalss_r32+1
2682func ss_phrase(prefix: *u8, h: *i64, terms: *i64, nterms: i64, kpout: *i64, klout: *i64, max: i64, exactout: *i64) -> i64 {
called by 3: dss_search_off_divg_phrase2main calls 8: sys_mmapss_manifest_dynss_terms_ordinalss_vrss_catss_readall+2
2839func ss_get_idx(prefix: *u8, key: *u8, ptrout: *i64, lenout: *i64) -> i64 {
called by 24: mainmainpj_putbl_cmd_loadvb_cmd_loadvc_put+18 calls 6: sys_mmapss_manifest_dynss_idx_lookupss_getss_catss_readall
2874func ss_get(prefix: *u8, key: *u8, ptrout: *i64, lenout: *i64) -> i64 {
called by 234: mainac_refac_lognac_log_strads_getads_add+228 calls 2: sys_mmapss_scan
2903func ss_cur_open(prefix: *u8) -> *i64
called by 1: do_retype calls 2: sys_mmapss_manifest_dyn
2917func ss_cur_next(prefix: *u8, st: *i64,
called by 1: do_retype calls 4: sys_mmapss_catss_readallss_r32
2986func ss_compact(prefix: *u8, segid: i64) -> i64 {
called by 6: mainmainmainmainmainmain calls 18: sys_mmapss_manifest_dynss_catss_readallss_r32ss_kcmp+12
3113func ss_compact_cap(prefix: *u8, segid: i64, cap: i64) -> i64 {
called by 5: mainmainmainmainmain calls 17: sys_mmapss_manifest_capss_catss_readallss_r32ss_kcmp+11