code wiki / (root) / nx_jsonl_writer.nx

nx_jsonl_writer.nx

buildroot/runtime/nx_jsonl_writer.nx

7607 B204 linesdepth 3pulls 3 transitivereach 1 importersview sourcekind library
docsdependenciesstructsconstsfunctions

about

nx_jsonl_writer.nx -- streaming JSONL output primitive. module: nishi-core.io.jsonl_writer depends: nishi-core.io.json_emit, nishi-core.io.syscalls disk_kb: 4 capability: CORE_IO license_tier: PUBLIC_NISHI_SUBSTRATE genealogy_id: rfc_7464_json_text_sequences + jsonlines_org_jsonl_specification + nishi_catalog_driven_cardinal_2026 Streaming line-delimited JSON output. One JSON object per line + newline terminator. Used by EVERY ingestion adapter for its canonical-record emit path (sources/usda_plants emits one row per PlantProfileResult, sources/kew_sid emits one per species, etc.). Per [[feedback-no-pdfs-no-proprietary-binary-formats]]: JSONL is the canonical open-format exchange format for the nishi data pipeline. Per [[feedback-bits-up-canonical-layer-no-reinventing]]: composes against nx_json_emit (shipped) for object serialization. ===== Output discipline ========================================== One JSON object per line. Line terminator = LF (0x0A). NO CRLF. NO leading whitespace. NO trailing whitespace before LF. NO pretty-print -- each record is one compact object. Round-trips through any RFC 7464 / jsonlines.org consumer. File-path convention (per nishi-library staging layout): nishi-library/staging/sources/<source>/<YYYY-MM-DD>/seq-<NNNNNN>.jsonl Each file caps at MAX_RECORDS_PER_FILE records (default 1000). Writer auto-rotates to next seq file when limit hit.

dependencies 2 imports · 1 importers

nx_syscalls.nx nx_json_emit.nx nx_jsonl_writer.nx nx_dead_letter_queue.nx

imports: nx_syscalls.nxnx_json_emit.nx

imported by: nx_dead_letter_queue.nx

structs

47struct JsonlWriter

consts

60const NX_JSONL_WRITER_BYTES: i64 = 80 // 10 fields * 8 bytes
61const NX_JSONL_DEFAULT_BUF_BYTES: i64 = 16384 // 16 KB scratch
62const NX_JSONL_DEFAULT_MAX_RECORDS: i64 = 1000
66const NX_JSONL_OK: i64 = 1
67const NX_JSONL_FD_INVALID: i64 = 2
68const NX_JSONL_BUFFER_OVERFLOW: i64 = 3
69const NX_JSONL_WRITE_FAIL: i64 = 4
70const NX_JSONL_NOT_OPEN: i64 = 5
71const NX_JSONL_ROTATION_FAILED: i64 = 6
168const NX_JSONL_SEQ_FILENAME_LEN: i64 = 16 // "seq-XXXXXX.jsonl"

functions

73func nx_jsonl_verdict_name(v: i64) -> *u8
85func nx_jsonl_writer_new() -> *JsonlWriter
calls 1: sys_mmap
106func nx_jsonl_writer_attach_fd(w: *JsonlWriter, fd: i64) -> i64
121func nx_jsonl_writer_emit(w: *JsonlWriter, record_len: i64, now_unix: i64) -> i64
calls 1: sys_write
149func nx_jsonl_writer_needs_rotation(w: *JsonlWriter) -> i64
155func nx_jsonl_writer_mark_rotated(w: *JsonlWriter) -> i64
170func nx_jsonl_writer_seq_filename(seq_n: i64, out: *u8, out_cap: i64) -> i64
198func nx_jsonl_writer_close(w: *JsonlWriter) -> i64
calls 1: sys_close