code wiki / (root) / nx_record.nx

nx_record.nx

buildroot/runtime/nx_record.nx

7924 B204 linesdepth 2pulls 2 transitivereach 358 importersview sourcekind librarytopic record
docsdependenciesstructsconstsfunctions

about

nx_record.nx -- NXR1: the TYPED, SELF-DESCRIBING record encoding for sovereign planes. WHY THIS EXISTS (debt seq1326). The TSV->plane migration moved the CONTAINER, not the ENCODING: nx_store_seed_lib stores each flat LINE as an opaque blob under a POSITIONAL key q:<seq> and is contractually required to rebuild the flat file byte-identically. So a "migrated" plane still holds tab-delimited untyped text. A plane-ified TSV is still a TSV. Its failure modes are structural, not sloppiness: a TAB inside a value corrupts the row; a record missing one column SHIFTS THE MEANING OF EVERY COLUMN AFTER IT (the live debt- plane already carries mixed 5-col and 7-col rows); there are no types, so every reader re-parses text and hopes; and there is no way to add a field without rewriting every existing record. NXR1 fixes all four AT THE REPRESENTATION, which is the only place they can be fixed: - FIELD IDS, not positions -> a missing field is simply absent. It cannot shift its neighbours. - LENGTH-PREFIXED values -> a tab, a newline, a NUL inside a value is just bytes. No escaping, no quoting, no delimiter to collide with. - EXPLICIT TYPES -> i64 is stored as i64, not as text a reader must trust. - SKIP-UNKNOWN + DEFAULTS -> a reader steps over field ids it does not know (forward compatible, rule 19 at the DATA layer), and an absent field reads as a caller default. That is Iceberg v3's default-value semantics: a new column costs zero backfill. LAYOUT (big-endian throughout, matching the seg-store's on-disk convention): record: ['N']['X']['R']['1'] [u16 nfields] then nfields x field field: [u16 field_id] [u8 type] [u32 len] [len bytes] This is a LEAF: it imports nx_syscalls ONLY. A record codec that had to pull in the storage engine just to read a u32 would be backwards -- the record layer sits UNDER the store, not beside it. license_tier: ORIGINAL No hw writes (Rule 26).

dependencies 1 imports · 2 importers

nx_syscalls.nx nx_record.nx nx_record_gate.nx nx_record_tsv.nx

imports: nx_syscalls.nx

imported by: nx_record_gate.nxnx_record_tsv.nx

structs

none

consts

31const NXR_MAGIC_N: i64 = 78
32const NXR_MAGIC_X: i64 = 88
33const NXR_MAGIC_R: i64 = 82
34const NXR_MAGIC_1: i64 = 49
35const NXR_HDR: i64 = 6 // magic(4) + u16 field count
36const NXR_FHDR: i64 = 7 // field id(2) + type(1) + len(4)
37const NXR_T_I64: i64 = 1
38const NXR_T_STR: i64 = 2
39const NXR_T_BYTES: i64 = 3
40const NXR_T_BOOL: i64 = 4
41const NXR_I64_BYTES: i64 = 8
42const NXR_ABSENT: i64 = 0
43const NXR_FOUND: i64 = 1
44const NXR_MAXF: i64 = 4096 // sanity bound on field count (fail-closed, never a silent cap)
45const NXR_OUTS_BYTES: i64 = 32 // scratch cell for the 3 out-params
46const NXR_BYTE_MASK: i64 = 255
47const NXR_SHIFT56: i64 = 56

functions

49func nxr_w16(p: *u8, off: i64, v: i64) -> i64
54func nxr_r16(p: *u8, off: i64) -> i64
59func nxr_w32(p: *u8, off: i64, v: i64) -> i64
called by 2: nxr_addnxr_add_i64
66func nxr_r32(p: *u8, off: i64) -> i64
called by 2: nxr_validnxr_find
73func nxr_w64(p: *u8, off: i64, v: i64) -> i64
called by 1: nxr_add_i64
81func nxr_r64(p: *u8, off: i64) -> i64
93func nxr_init(b: *u8) -> i64
called by 3: mainrtv_to_nxrmain calls 1: nxr_w16
104func nxr_add(b: *u8, off: i64, fid: i64, ty: i64, val: *u8, vlen: i64) -> i64
121func nxr_add_i64(b: *u8, off: i64, fid: i64, v: i64) -> i64
132func nxr_count(b: *u8) -> i64 { return nxr_r16(b, 4) }
137func nxr_valid(b: *u8, n: i64) -> i64
called by 2: nxr_findmain calls 2: nxr_r16nxr_r32
167func nxr_find(b: *u8, n: i64, fid: i64, outs: *i64) -> i64
194func nxr_get_i64(b: *u8, n: i64, fid: i64, defval: i64) -> i64