nx_jsonl_writer.nx
buildroot/runtime/nx_jsonl_writer.nx
about
nx_jsonl_writer.nx -- streaming JSONL output primitive.
module: nishi-core.io.jsonl_writer
depends: nishi-core.io.json_emit, nishi-core.io.syscalls
disk_kb: 4
capability: CORE_IO
license_tier: PUBLIC_NISHI_SUBSTRATE
genealogy_id: rfc_7464_json_text_sequences +
jsonlines_org_jsonl_specification +
nishi_catalog_driven_cardinal_2026
Streaming line-delimited JSON output. One JSON object per line +
newline terminator. Used by EVERY ingestion adapter for its
canonical-record emit path (sources/usda_plants emits one row per
PlantProfileResult, sources/kew_sid emits one per species, etc.).
Per [[feedback-no-pdfs-no-proprietary-binary-formats]]: JSONL is
the canonical open-format exchange format for the nishi data
pipeline. Per [[feedback-bits-up-canonical-layer-no-reinventing]]:
composes against nx_json_emit (shipped) for object serialization.
===== Output discipline ==========================================
One JSON object per line. Line terminator = LF (0x0A). NO CRLF.
NO leading whitespace. NO trailing whitespace before LF. NO
pretty-print -- each record is one compact object. Round-trips
through any RFC 7464 / jsonlines.org consumer.
File-path convention (per nishi-library staging layout):
nishi-library/staging/sources/<source>/<YYYY-MM-DD>/seq-<NNNNNN>.jsonl
Each file caps at MAX_RECORDS_PER_FILE records (default 1000).
Writer auto-rotates to next seq file when limit hit.
dependencies 2 imports · 1 importers
imports: nx_syscalls.nxnx_json_emit.nx
imported by: nx_dead_letter_queue.nx
structs
| 47 | struct JsonlWriter |
consts
| 60 | const NX_JSONL_WRITER_BYTES: i64 = 80 // 10 fields * 8 bytes |
| 61 | const NX_JSONL_DEFAULT_BUF_BYTES: i64 = 16384 // 16 KB scratch |
| 62 | const NX_JSONL_DEFAULT_MAX_RECORDS: i64 = 1000 |
| 66 | const NX_JSONL_OK: i64 = 1 |
| 67 | const NX_JSONL_FD_INVALID: i64 = 2 |
| 68 | const NX_JSONL_BUFFER_OVERFLOW: i64 = 3 |
| 69 | const NX_JSONL_WRITE_FAIL: i64 = 4 |
| 70 | const NX_JSONL_NOT_OPEN: i64 = 5 |
| 71 | const NX_JSONL_ROTATION_FAILED: i64 = 6 |
| 168 | const NX_JSONL_SEQ_FILENAME_LEN: i64 = 16 // "seq-XXXXXX.jsonl" |
functions
| 73 | func nx_jsonl_verdict_name(v: i64) -> *u8 |
| 85 | func nx_jsonl_writer_new() -> *JsonlWriter calls 1: sys_mmap |
| 106 | func nx_jsonl_writer_attach_fd(w: *JsonlWriter, fd: i64) -> i64 |
| 121 | func nx_jsonl_writer_emit(w: *JsonlWriter, record_len: i64, now_unix: i64) -> i64 calls 1: sys_write |
| 149 | func nx_jsonl_writer_needs_rotation(w: *JsonlWriter) -> i64 |
| 155 | func nx_jsonl_writer_mark_rotated(w: *JsonlWriter) -> i64 |
| 170 | func nx_jsonl_writer_seq_filename(seq_n: i64, out: *u8, out_cap: i64) -> i64 |
| 198 | func nx_jsonl_writer_close(w: *JsonlWriter) -> i64 calls 1: sys_close |