code wiki / (root) / nx_colframe.nx

nx_colframe.nx

buildroot/runtime/nx_colframe.nx

5276 B104 linesdepth 2pulls 2 transitivereach 9 importersview sourcekind library
docsdependenciesstructsconstsfunctions

about

nx_colframe.nx -- SOVEREIGN ZERO-COPY COLUMNAR INTERCHANGE (arrow-interchange, F-arrow). The Arrow idea, built from the first byte up as our OWN format, not a clone: a single self-describing contiguous buffer that carries N columns COLUMNAR (each column packed contiguously, 8-byte aligned), so any organ that receives the buffer reads a whole column with ONE pointer cast -- zero per-value deserialization, zero copy on the read side. Cross-organ dataset passing becomes "hand over a pointer + a length". THE EXACT BYTE LAYOUT (all fields native i64 little-endian; the value IS the layout): [0] magic = CF_MAGIC (identifies + versions the frame) [1] version = CF_VERSION [2] ncol [3] nrows then a SCHEMA table of ncol entries, 3 i64 each (i64-index 4 + i*3): [+0] type (CF_T_I64 = 0; the v1 type) [+1] name_off (byte offset to the column's NUL-terminated name) [+2] data_off (byte offset to the column's data; 8-BYTE ALIGNED so the *i64 cast is valid+fast) then the packed name strings, then each column's nrows*8 bytes of i64 data. ZERO-COPY is provable, not asserted: cf_col(buf,i) returns buf+data_off -- a pointer INTO the shared buffer. Called twice it returns the SAME address (a view, never a per-call materialisation), and an analytics organ runs DIRECTLY on that view. EXCEED vs Arrow: 100% integer/fixed layout => bit-identical frames across runs (Arrow's buffers carry padding/impl variance). license_tier: ORIGINAL No hw writes.

dependencies 1 imports · 5 importers

nx_syscalls.nx nx_colframe.nx nx_colframe_gate.nx nx_distexec.nx nx_mrshuffle.nx nx_parexec.nx nx_vecexec.nx

imports: nx_syscalls.nx

imported by: nx_colframe_gate.nxnx_distexec.nxnx_mrshuffle.nxnx_parexec.nxnx_vecexec.nx

structs

none

consts

24const CF_MAGIC: i64 = 5136462849 // "NXCF" epoch tag; distinctive, non-zero
25const CF_VERSION: i64 = 1
26const CF_T_I64: i64 = 0
27const CF_HDR_I64: i64 = 4 // magic,version,ncol,nrows
28const CF_SCH_I64: i64 = 3 // per-column schema words

functions

30func cf_align8(x: i64) -> i64 { let r: i64 = x % 8; if r == 0 { return x } return x + (8 - r) }
31func cf_slen(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } return n }
called by 1: cf_encoded_bytes
34func cf_encoded_bytes(names: *i64, ncol: i64, nrows: i64) -> i64
46func cf_encode(cols: *i64, names: *i64, ncol: i64, nrows: i64, out: *u8) -> i64
83func cf_valid(buf: *u8) -> i64 { let w: *i64 = buf as *i64; if w[0] == CF_MAGIC { if w[1] == CF_VERSION { return 1 } } return 0 }
called by 1: main
84func cf_ncol(buf: *u8) -> i64 { let w: *i64 = buf as *i64; return w[2] }
called by 2: cf_col_indexmain
85func cf_nrows(buf: *u8) -> i64 { let w: *i64 = buf as *i64; return w[3] }
86func cf_type(buf: *u8, i: i64) -> i64 { let w: *i64 = buf as *i64; return w[CF_HDR_I64 + i * CF_SCH_I64] }
called by 1: main
87func cf_name(buf: *u8, i: i64) -> *u8 { let w: *i64 = buf as *i64; let off: i64 = w[CF_HDR_I64 + i * CF_SCH_I64 + 1]; return (((buf as i64) + off) as *u8) }
called by 1: cf_col_index
88func cf_data_off(buf: *u8, i: i64) -> i64 { let w: *i64 = buf as *i64; return w[CF_HDR_I64 + i * CF_SCH_I64 + 2] }
called by 2: cf_colmain
90func cf_col(buf: *u8, i: i64) -> *i64 { return (((buf as i64) + cf_data_off(buf, i)) as *i64) }
92func cf_col_index(buf: *u8, name: *u8) -> i64
called by 1: main calls 2: cf_ncolcf_name