nx_record.nx
buildroot/runtime/nx_record.nx
about
nx_record.nx -- NXR1: the TYPED, SELF-DESCRIBING record encoding for sovereign planes.
WHY THIS EXISTS (debt seq1326). The TSV->plane migration moved the CONTAINER, not the ENCODING:
nx_store_seed_lib stores each flat LINE as an opaque blob under a POSITIONAL key q:<seq> and is
contractually required to rebuild the flat file byte-identically. So a "migrated" plane still holds
tab-delimited untyped text. A plane-ified TSV is still a TSV. Its failure modes are structural, not
sloppiness: a TAB inside a value corrupts the row; a record missing one column SHIFTS THE MEANING OF
EVERY COLUMN AFTER IT (the live debt- plane already carries mixed 5-col and 7-col rows); there are no
types, so every reader re-parses text and hopes; and there is no way to add a field without rewriting
every existing record.
NXR1 fixes all four AT THE REPRESENTATION, which is the only place they can be fixed:
- FIELD IDS, not positions -> a missing field is simply absent. It cannot shift its neighbours.
- LENGTH-PREFIXED values -> a tab, a newline, a NUL inside a value is just bytes. No escaping,
no quoting, no delimiter to collide with.
- EXPLICIT TYPES -> i64 is stored as i64, not as text a reader must trust.
- SKIP-UNKNOWN + DEFAULTS -> a reader steps over field ids it does not know (forward compatible,
rule 19 at the DATA layer), and an absent field reads as a caller
default. That is Iceberg v3's default-value semantics: a new column
costs zero backfill.
LAYOUT (big-endian throughout, matching the seg-store's on-disk convention):
record: ['N']['X']['R']['1'] [u16 nfields] then nfields x field
field: [u16 field_id] [u8 type] [u32 len] [len bytes]
This is a LEAF: it imports nx_syscalls ONLY. A record codec that had to pull in the storage engine
just to read a u32 would be backwards -- the record layer sits UNDER the store, not beside it.
license_tier: ORIGINAL No hw writes (Rule 26).
dependencies 1 imports · 2 importers
imports: nx_syscalls.nx
imported by: nx_record_gate.nxnx_record_tsv.nx
structs
| none |
consts
| 31 | const NXR_MAGIC_N: i64 = 78 |
| 32 | const NXR_MAGIC_X: i64 = 88 |
| 33 | const NXR_MAGIC_R: i64 = 82 |
| 34 | const NXR_MAGIC_1: i64 = 49 |
| 35 | const NXR_HDR: i64 = 6 // magic(4) + u16 field count |
| 36 | const NXR_FHDR: i64 = 7 // field id(2) + type(1) + len(4) |
| 37 | const NXR_T_I64: i64 = 1 |
| 38 | const NXR_T_STR: i64 = 2 |
| 39 | const NXR_T_BYTES: i64 = 3 |
| 40 | const NXR_T_BOOL: i64 = 4 |
| 41 | const NXR_I64_BYTES: i64 = 8 |
| 42 | const NXR_ABSENT: i64 = 0 |
| 43 | const NXR_FOUND: i64 = 1 |
| 44 | const NXR_MAXF: i64 = 4096 // sanity bound on field count (fail-closed, never a silent cap) |
| 45 | const NXR_OUTS_BYTES: i64 = 32 // scratch cell for the 3 out-params |
| 46 | const NXR_BYTE_MASK: i64 = 255 |
| 47 | const NXR_SHIFT56: i64 = 56 |
functions
| 49 | func nxr_w16(p: *u8, off: i64, v: i64) -> i64 |
| 54 | func nxr_r16(p: *u8, off: i64) -> i64 |
| 59 | func nxr_w32(p: *u8, off: i64, v: i64) -> i64 |
| 66 | func nxr_r32(p: *u8, off: i64) -> i64 |
| 73 | func nxr_w64(p: *u8, off: i64, v: i64) -> i64 called by 1: nxr_add_i64 |
| 81 | func nxr_r64(p: *u8, off: i64) -> i64 |
| 93 | func nxr_init(b: *u8) -> i64 |
| 104 | func nxr_add(b: *u8, off: i64, fid: i64, ty: i64, val: *u8, vlen: i64) -> i64 |
| 121 | func nxr_add_i64(b: *u8, off: i64, fid: i64, v: i64) -> i64 |
| 132 | func nxr_count(b: *u8) -> i64 { return nxr_r16(b, 4) } |
| 137 | func nxr_valid(b: *u8, n: i64) -> i64 |
| 167 | func nxr_find(b: *u8, n: i64, fid: i64, outs: *i64) -> i64 |
| 194 | func nxr_get_i64(b: *u8, n: i64, fid: i64, defval: i64) -> i64 |