code wiki / _hdl_build / nx_galx_cidput.nx
nx_galx_cidput.nx
buildroot/runtime/_hdl_build/nx_galx_cidput.nx
about
nx_galx_cidput.nx -- NATIVE incremental insert into the sovereign cid->path store. No TSV.
Operator 2026-08-07: *"we shouldnt be using tsvs thats probably why it should be nishi native"*.
Correct, and the estate had already said so: nx_galx_cid_index's own header reads *"The .idx +
.blob ARE the runtime store; the TSV is only a one-time build input (and can then be retired)."*
It was never retired -- because the store shipped READ-ONLY. gs_cidindex_build() can only build
the whole thing FROM a TSV, and there was no way to add one entry. So every addition fell back
to appending 70 bytes of text to a 34MB file and recompiling all 235,445 rows.
★★★★★★ A STORE YOU CAN ONLY BUILD, NEVER APPEND TO, KEEPS ITS OWN LEGACY INPUT FORMAT ALIVE
FOREVER -- the "retired" format is load-bearing precisely because the replacement has no writer.
★ THE MISSING PRIMITIVE WAS NOT THE INDEX, IT WAS INSERT.
The format already supports O(1) insert; nobody had written it. Open-addressed, fixed buckets:
idx: [8 n_buckets][8 n_entries] then n_buckets * 24: [8 cid_hash][8 path_off][8 path_len] (big-endian)
blob: concatenated path bytes
So an insert is: append the path to blob, linear-probe to an empty bucket, write 24 bytes, bump
n_entries. Three small writes at known offsets -- sys_lseek + sys_openat_rdwr, no file rewrite.
⚠LOAD FACTOR IS A HARD LIMIT, NOT A SUGGESTION. n_buckets is fixed at build time; an open-
addressed table that passes ~0.5 degrades toward O(n) and at 1.0 never terminates its probe.
This REFUSES past the builder's own nent*2 <= nb rule and names the remedy (rebuild, which
re-sizes), because ★ A GUARD THAT CANNOT BE SATISFIED PRODUCES A BYPASS.
license_tier: ORIGINAL
dependencies 2 imports · 4 importers
imports: nx_syscalls.nxnx_galx_cid_index.nx
imported by: nx_gallery_serve.nxnx_galx_put.nxnx_galx_repair.nxnx_gen_ingest.nx
structs
| none |
consts
| 28 | const CP_SEEK_SET: i64 = 0 |
| 29 | const CP_SEEK_END: i64 = 2 |
| 32 | const CP_INSERTED: i64 = 1 |
| 33 | const CP_PRESENT: i64 = 0 |
| 34 | const CP_EIO: i64 = 0 - 1 |
| 35 | const CP_EFULL: i64 = 0 - 2 // load factor exhausted -> rebuild to resize |
| 36 | const CP_EBADIDX: i64 = 0 - 3 |
functions
| 38 | func cp_w64(b: *u8, o: i64, v: i64) -> i64 called by 1: nx_galx_hashput |
| 49 | func cp_len(s: *u8) -> i64 called by 1: nx_galx_hashput |
| 58 | func nx_galx_hashput(idx_path: *u8, blob_path: *u8, h_in: i64, path: *u8, plen_in: i64) -> i64 called by 2: nx_galx_cidputgs_cidindex_build_preserving calls 9: sys_openat_rdwrsys_mmapsys_readsys_closecidx_rd64sys_lseek+3 |
| 125 | func nx_galx_cidput(idx_path: *u8, blob_path: *u8, cid: *u8, path: *u8) -> i64 |
| 139 | func gs_cidindex_build_preserving(tsv_path: *u8, idx_path: *u8, blob_path: *u8) -> i64 |