code wiki / _hdl_build / nx_claude_harvest.nx

nx_claude_harvest.nx

buildroot/runtime/_hdl_build/nx_claude_harvest.nx

197776 B4001 linesdepth 6pulls 17 transitivereach 12 importersview sourcekind tooltopic claude
docsdependenciesstructsconstsfunctions

about

nx_claude_harvest.nx -- CLAUDE TRANSCRIPT HARVESTER + OUTSIDE-GAUGE (miner-sota lane F964/F973, 2026-07-23). PORTABLE organ (gen-image pattern: runs where the transcripts live -- laptop WSL today, NAS, any host): parses Claude Code transcript JSONL into sovereign actlog frames, bulk-ingests staged frames into a journal (validated, marker-idempotent), and derives the outside_permil gauge (nishi vs outside tool split) that MUST FALL as capabilities move inside (MG00 baseline 651, 2026-07-23). Composes nx_sovjson_lib (sj_*) -- no helper dup (D001 discipline). State derived; journals append-only (O_APPEND, conflict-free, ws_sync lineage). No hw writes (Rule 26). frames <transcript.jsonl> <ws> <outjournal> -> parse transcript, append actlog frames; HARVEST OK ingest <framesfile> <journal> -> validated bulk append, idempotent by marker frame gauge <journal> [days] -> JSON outside_permil (0 days = all frames) beat <chbeat.conf> -> DM1 session->NAS capture beat (devmgmt rung, 2026-08-20): walk EVERY transcript under root (all depths, getdents looped until 0), ch_delta each against a per-transcript byte HWM, split the fresh frames into ts-day PIECES sized under the MCP body cliff, ingest each piece (ch_ingest, marker-idempotent) into per-day journals -- the local twin always, the NAS landing via the forked wire when push=1. Pieces are the chunk unit AND the ingest unit, so the pipeline is resumable at every stage with no append primitive over the wire. Counters partition and SUM; unknown head-types bucket UNKNOWN with a named worklist. selftest <scratch-prefix> -> gate T1..T8 + beat teeth (unique per-run scratch names) SSOT for this source is buildroot/runtime/_hdl_build/ on the NAS; the laptop nxc2 tree is a build twin (synced per edit, video-lane pattern) because the beat RUNS WHERE THE TRANSCRIPTS LIVE (laptop WSL). license_tier: ORIGINAL

dependencies 9 imports · 7 importers

nx_sovjson_lib.nx nx_json.nx nx_utf8.nx nx_sha256.nx nx_fio.nx nx_syscalls.nx nx_gate_verdict.nx nx_tool_run.nx nx_mccabe_lib.nx nx_claude_harvest.nx nx_claude_requests_canonical_gate_ nx_claude_usage_adopt_gate_t76.nx nx_claude_usage_gate_t76.nx nx_usage_artifact_gate_t173.nx nx_usage_artifact_t173.nx nx_wfcost.nx nx_wfcost_outcomes_t144.nx

imports: nx_sovjson_lib.nxnx_json.nxnx_utf8.nxnx_sha256.nxnx_fio.nxnx_syscalls.nxnx_gate_verdict.nxnx_tool_run.nxnx_mccabe_lib.nx

imported by: nx_claude_requests_canonical_gate_t144.nxnx_claude_usage_adopt_gate_t76.nxnx_claude_usage_gate_t76.nxnx_usage_artifact_gate_t173.nxnx_usage_artifact_t173.nxnx_wfcost.nxnx_wfcost_outcomes_t144.nx

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sj_lit_eq sj_vlen sj_puts sys_write ch_usage_requests_file sha256_workspace_bytes sha256_context_aligned_byt sys_mmap_shared sha256_init_workspace sha256_workspace_bytes ↻ sha256_context_aligned_byt ↻ sha256_seed_allocated sha256_k sj_vlen ↻ sys_openat_exclusive ch_usage_window_sink sys_mmap nxa_die sys_write ↻ sys_exit nxa_lock_take nxa_lock_addr sys_write ↻ nxa_lock_give nxa_lock_addr ↻ nxa_report_overrun sys_write ↻ nxa_dump_printable sys_write ↻ nxa_dump_sizes sys_write ↻ sj_vlen ↻ chu_emit_to sys_mmap ↻ sj_cat sj_catn sys_mmap ↻ sj_vlen ↻ sys_now_realtime_sec

structs

none

consts

35const CH_MAGIC_146097: i64 = 146097
36const CH_MAGIC_719468: i64 = 719468
37const CH_MAGIC_86400: i64 = 86400
38const CH_MAGIC_3600: i64 = 3600
39const CH_MAGIC_1784764900: i64 = 1784764900
40const CH_MAGIC_1784764901: i64 = 1784764901
41const CH_MAGIC_1784764803: i64 = 1784764803
43const CH_WIN: i64 = 67108864
44const CH_JWIN: i64 = 4194304
45const CH_OUT: i64 = 65536
46const CH_LINE: i64 = 4096
47const CH_NAME_MAX: i64 = 120
48const CH_NAME_SCAN: i64 = 400
49const CH_MODE_644: i64 = 420
51const CH_AH_MAXPATHS: i64 = 8192
52const CH_AH_PATHCELL: i64 = 512
53const CH_AH_MAXSESS: i64 = 1024
54const CH_AH_SESSCELL: i64 = 64
55const CH_AH_CTRS: i64 = 32
56const CH_AH_SPINE: *u8 = "knowledge/status/mccabe.spine"
57const CH_AH_STATUS: *u8 = "knowledge/status/author_health.status"
58const CH_AH_TREE: *u8 = "buildroot/runtime/"
59const CH_AH_RC_UNOBS: i64 = 3
60const CH_AH_RC_NOPATH: i64 = 4
61const CH_AH_FRAMES: i64 = 0
62const CH_AH_PATHFRAMES: i64 = 1
63const CH_AH_UNIQUE: i64 = 2
64const CH_AH_INTREE: i64 = 3
65const CH_AH_OUTSIDE: i64 = 4
66const CH_AH_FIRSTTS: i64 = 5
67const CH_AH_SESSIONS: i64 = 6
68const CH_AH_FILES: i64 = 7
69const CH_AH_FUNCS: i64 = 8
70const CH_AH_SUMVG: i64 = 9
71const CH_AH_ABOVE: i64 = 10
72const CH_AH_DFILES: i64 = 11
73const CH_AH_DFUNCS: i64 = 12
74const CH_AH_DABOVE: i64 = 13
75const CH_AH_DTS: i64 = 14
76const CH_AH_UNREAD: i64 = 15
77const CH_AH_OVERFLOW: i64 = 16
78const CH_AH_FIX_TS: i64 = 1788600000
79const CH_AH_FIX_DFILES: i64 = 10
80const CH_AH_FIX_DFUNCS: i64 = 20
81const CH_AH_FIX_DABOVE: i64 = 3
82const CH_CH_D0: i64 = 48
83const CH_CH_D9: i64 = 57
84const CH_DEC: i64 = 10
87const CH_BT_DIRWIN: i64 = 65536 // getdents64 batch buffer -- LOOPED until 0, never a listing cap
88const CH_BT_PATHMAX: i64 = 512 // one path cell; longest real transcript path measured ~120 B, 4x headroom
89const CH_BT_MAXFILES: i64 = 16384 // walk table capacity (837 files measured 2026-08-20); hitting it REFUSES loudly
90const CH_BT_MAXDIRS: i64 = 4096
91const CH_BT_ARENA: i64 = 8388608 // path arena = CH_BT_MAXFILES * CH_BT_PATHMAX exactly
92const CH_BT_TYPEMAX: i64 = 64 // distinct head-type slots; overflow announces FLOOR-NOT-TOTAL
93const CH_BT_DAYMAX: i64 = 128 // distinct ts-days per run (backfill spans ~60); overflow REFUSES loudly
94const CH_BT_DAYCELL: i64 = 16 // one day-name cell: YYYYMMDD (8) or tszero (6) + NUL, padded
95const CH_BT_DEFAULT_WIRE_TIMEOUT_MS:i64=120000 // bootstrap fallback; wire_timeout_ms config overrides
96const CH_MCP_BODY_CLIFF: i64 = 65536 // measured: the MCP write body silently drops beyond ~64 KiB
97const CH_BT_CTRS: i64 = 32 // bcx slot count (map at ch_beat)
98const CH_MODE_755: i64 = 493 // rwxr-xr-x for the beat's own state/spool/journal dirs
99const CH_MAGIC_1460: i64 = 1460 // 4*365 (Hinnant civil-from-days)
100const CH_MAGIC_36524: i64 = 36524 // 100*365 + 24 (Hinnant)
101const CH_MAGIC_146096: i64 = 146096 // CH_MAGIC_146097 - 1 (Hinnant)
222const CH_PATH_SCAN: i64 = 4096 // one transcript line can carry the write's CONTENT before its path key; the frame line is CH_LINE anyway
223const CH_PATH_MAX: i64 = 512
224const CH_CH_BSL: i64 = 92
225const CH_WK_NONE: i64 = 0
226const CH_WK_ARGV: i64 = 1
227const CH_WK_FILEPATH: i64 = 2
527const CH_WS_MAX: i64 = 4096
790const CH_CLS_MAX: i64 = 512 // distinct outside tool names tracked (transcript tool names measured ~60 distinct)
791const CH_CLS_NAME: i64 = 64 // one name cell; longest builtin tool name measured well under this
792const CH_CLS_SHOW: i64 = 12 // JSON shows this many, the rest fold into other
793const CH_CLS_JSHOW: i64 = 6 // the journal row shows this many, the rest fold into other
1863const CH_MK_MAX: i64 = 64
1864const CH_CONF_WIN: i64 = 65536
1865const CH_TXTWIN: i64 = 262144
3347const CHU_SAFE:i64=9007199254740991 // exact JSON integer interoperability limit
3348const CHU_REQUIRED_FIELDS:i64=32767 // bits0..14: exact observed request schema
3349const CHU_ROW:i64=21 // mask, kind, six identity spans, six counters, epoch
3647const CHU_SNAPSHOT_MODE:i64=384 // POSIX owner read/write, no group or world access
3919const CHUA_HEADER:*u8="schema|state|code|scan_sha256|start_inclusive_epoch|end_exclusive_epoch|totals|unique_requests|snapshot_unique_requests|duplicate_records|ignored_records"
3920const CHUA_REQUEST:*u8="schema|scan_sha256|sequence|timestamp|thread_id|response_id|turn_id|session_id|root_turn_id|usage|provider|requested_model|actual_model|role|task_id|attempt_id|review_id|accepted_deployed_outcome|elapsed_ms|billed_spend|attribution"
3921const CHUA_END:*u8="schema|scan_sha256|emitted_requests|expected_requests|complete"
3922const CHUA_COUNTERS:*u8="input_tokens|cached_input_tokens|cache_write_input_tokens|output_tokens|reasoning_output_tokens|total_tokens"

functions

103func ch_read(path: *u8, buf: *u8, cap: i64) -> i64
113func ch_find_from(q: *u8, from: i64, endlim: i64, lit: *u8) -> i64
131func ch_find(q: *u8, n: i64, lit: *u8) -> i64
136func ch_starts(q: *u8, s: i64, e: i64, lit: *u8) -> i64
145func ch_digits(q: *u8, s: i64, e: i64) -> i64
152func ch_epoch_ymdhms(y: i64, mo: i64, d: i64, h: i64, mi: i64, s: i64) -> i64
called by 1: ch_iso_epoch
164func ch_iso_epoch(q: *u8, s: i64, lim: i64) -> i64
190func ch_frame_write(fd: i64, ln: *u8, ts: i64, ws: *u8, tool: *u8, verb: *u8, outcome: *u8, note: *u8) -> i64
207func ch_frame_append(path: *u8, ts: i64, ws: *u8, tool: *u8, verb: *u8, outcome: *u8, note: *u8) -> i64
228func ch_tool_writer_kind(nm: *u8) -> i64
called by 1: ch_frame_note calls 2: sj_vlensj_lit_eq
236func ch_json_str_copy(buf: *u8, k0: i64, le: i64, pb: *u8, cap: i64) -> i64
called by 1: ch_tool_path
253func ch_tool_path(buf: *u8, np: i64, le: i64, kind: i64, pb: *u8) -> i64
275func ch_frame_note(buf: *u8, np: i64, le: i64, nm: *u8, verb: *u8, pb: *u8, nb: *u8) -> *u8
288func ch_frames(tpath: *u8, ws: *u8, outj: *u8, tcx: *i64) -> i64
350func ch_hwm_read(path: *u8) -> i64
362func ch_hwm_write(path: *u8, v: i64) -> i64
372func ch_delta(tpath: *u8, ws: *u8, outj: *u8, hwmpath: *u8, hcx: *i64) -> i64
451func ch_ingest(fpath: *u8, jpath: *u8, mcx: *i64) -> i64
506func ch_nishi_tool(q: *u8, s: i64, e: i64) -> i64
513func ch_key(d: *u8, o: i64, name: *u8) -> i64
528func ch_bg_intern(q: *u8, cs: i64, ce: i64, wss: *i64, wse: *i64, np: *i64) -> i64
called by 1: ch_bootgauge
545func ch_bootgauge(journal: *u8, k: i64, d: *u8) -> i64
622func ch_gauge_calc(jpath: *u8, days: i64, g: *i64) -> i64
689func ch_gauge_json(jpath: *u8, days: i64, d: *u8) -> i64
737func ch_page(jpath: *u8, outp: *u8, days: i64) -> i64
794func ch_classes_calc(jpath: *u8, names: *u8, counts: *i64, c: *i64) -> i64
846func ch_classes_top(counts: *i64, used: *i64, ncls: i64) -> i64
called by 1: ch_classes_cat
852func ch_classes_cat(d: *u8, o0: i64, names: *u8, counts: *i64, ncls: i64, show: i64, outside: i64) -> i64
876func ch_classes_json(jpath: *u8, d: *u8) -> i64
904func ch_classes(jpath: *u8, d: *u8) -> i64 { return ch_classes_json(jpath, d) }
called by 1: main calls 1: ch_classes_json
910func ch_journal(jpath: *u8, planpath: *u8, rung: *u8) -> i64
969func ch_allow_path(name: *u8, outp: *u8) -> i64 { return sj_allow_path(name, outp) }
called by 2: ch_proposech_selftest calls 1: sj_allow_path
970func ch_propose(jpath: *u8, ipath: *u8, thr: i64, filer: *u8, cout: *u8, colen: *i64) -> i64
1008func ch_selftest(prefix: *u8) -> i64
1869func ch_run_of(q: *u8, s: i64, e: i64, g: *u8) -> i64
called by 1: ch_rank_text calls 1: sj_vlen
1898func ch_conf_load(path: *u8, cbuf: *u8, mk: *i64, sc: *i64) -> i64
1945func ch_isalpha(c: u8) -> i64
called by 1: ch_rank_text
1954func ch_rank_text(q: *u8, s: i64, e: i64, cbuf: *u8, mk: *i64, sc: *i64) -> i64
1984func ch_learn(tpath: *u8, ws: *u8, outp: *u8, confp: *u8, cx: *i64) -> i64
2113func ch_bt_zeq(a: *u8, b: *u8) -> i64
2118func ch_bt_ends(s: *u8, sfx: *u8) -> i64
called by 1: ch_bt_walk calls 2: sj_vlensj_lit_eq
2124func ch_bt_le16(b: *u8, off: i64) -> i64 { return (b[off] as i64) + (b[off+1] as i64) * 256 }
called by 1: ch_bt_stat
2125func ch_bt_le64(b: *u8, off: i64) -> i64
called by 1: ch_bt_stat
2135func ch_bt_stat(path: *u8, sb: *u8, k: *i64) -> i64
2149func ch_bt_walk(root: *u8, sfx: *u8, ar: *u8, fo: *i64, wcx: *i64) -> i64
2233func ch_bt_day(ts: i64, d: *u8) -> i64
called by 1: ch_bt_split calls 1: sj_cat
2259func ch_bt_kv(cbuf: *u8, i: i64, le: i64, key: *u8) -> i64
called by 1: ch_bt_conf calls 2: ch_startssj_vlen
2268func ch_bt_conf(path: *u8, cbuf: *u8, off: *i64, iv: *i64, kt: *i64, kcx: *i64) -> i64
2360func ch_bt_census(tpath: *u8, from: i64, to: i64, cbuf: *u8, kt: *i64, kn: i64, ua: *u8, uo: *i64, ucx: *i64, ccx: *i64) -> i64
2427func ch_bt_path2(out: *u8, a: *u8, b: *u8) -> i64
called by 1: ch_beat calls 1: sj_cat
2435func ch_bt_base(path: *u8) -> *u8
2441func ch_bt_stem(path: *u8, out: *u8) -> i64
called by 1: ch_beat calls 2: ch_bt_basesj_vlen
2451func ch_bt_hwmpath(state: *u8, rel: *u8, out: *u8) -> i64
called by 1: ch_beat calls 1: sj_cat
2469func ch_bt_lines(path: *u8) -> i64
2485func ch_bt_int_after(b: *u8, n: i64, key: *u8) -> i64
2504func ch_bt_parse_sha(b: *u8, n: i64, out: *u8) -> i64
2527func ch_bt_wire(wire: *u8, script: *u8, tool: *u8, cap: *u8, a1: *u8, a2: *u8, a3: *u8, a4: *u8, tmo: i64, cout: *u8, colen: *i64) -> i64
2541func ch_bt_flushp(spool: *u8, day: *u8, stamp: i64, seq: i64, cell: *u8, clen: i64) -> i64
2567func ch_bt_split(rawpath: *u8, spool: *u8, stamp: i64, piece_raw: i64, dcx: *i64) -> i64
2646func ch_bt_pieceday(pbase: *u8, db: *u8) -> i64
2677func ch_bt_pushreason(wrote_ok: i64, okr: i64, same: i64) -> *u8
2689func ch_bt_push1(ppath: *u8, cbuf: *u8, off: *i64, iv: *i64, pjr: *u8, bcx: *i64) -> i64
2791func ch_beat(confpath: *u8, bcx: *i64) -> i64
3132func ch_ah_prefix(p: *u8, pre: *u8) -> i64
called by 1: ce_author_health
3138func ch_ah_field_n(buf: *u8, i: i64, le: i64, want: i64, out: *u8, cap: i64) -> i64
called by 1: ce_author_health
3162func ch_ah_find(tab: *u8, n: i64, cell: i64, p: *u8) -> i64
called by 1: ce_author_health calls 2: sj_lit_eqsj_vlen
3171func ch_ah_lastline(buf: *u8, n: i64) -> i64
called by 1: ce_author_health calls 1: sj_le
3179func ch_ah_num(buf: *u8, p0: i64, to: i64) -> i64
3191func ch_ah_field(buf: *u8, from: i64, to: i64, key: *u8) -> i64
3196func ce_author_health(jpath: *u8, pre: *u8, spine: *u8, cx: *i64, ob: *u8) -> i64
3350func chu_uint(q:*u8,s:i64,n:i64)->i64
3356func chu_key(t:*NxJsonTok,ctx:i64)->i64
called by 1: chu_value calls 1: sj_lit_eq
3381func chu_value(t:*NxJsonTok,ctx:i64,r:*i64,depth:i64)->i64
3431func chu_row(t:*NxJsonTok,q:*u8,n:i64,r:*i64)->i64
3457func chu_span_same(q:*u8,a:*i64,b:*i64,slot:i64)->i64
called by 1: chu_totals
3462func chu_totals(q:*u8,n:i64,from:i64,to:i64,table:*i64,t:*NxJsonTok,r:*i64,c:*i64)->i64
3502func chu_write_hash(fd:i64,q:*u8,n:i64,hash:*Sha256)->i64
3507func chu_write_all(fd:i64,q:*u8,n:i64)->i64{return chu_write_hash(fd,q,n,0 as *Sha256)}
3508func chu_request_records_to(fd:i64,hash:*Sha256,q:*u8,table:*i64,from:i64,to:i64,c:*i64,digest:*u8)->i64
3543func chu_request_records(fd:i64,q:*u8,table:*i64,from:i64,to:i64,c:*i64,digest:*u8)->i64{return chu_request_records_to(fd,0 as *Sha256,q,table,from,to,c,digest)}
called by 1: main calls 1: chu_request_records_to
3544func chu_emit_to(fd:i64,hash:*Sha256,path:*u8,digest:*u8,state:*u8,rc:i64,extent:i64,start:i64,end:i64,head:i64,tail:i64,from:i64,to:i64,c:*i64)->i64
3580func chu_emit(path:*u8,digest:*u8,state:*u8,rc:i64,extent:i64,start:i64,end:i64,head:i64,tail:i64,from:i64,to:i64,c:*i64)->i64{return chu_emit_to(1,0 as *Sha256,path,digest,state,rc,extent,start,end,head,tail,from,to,c)}
calls 1: chu_emit_to
3581func ch_usage_window_sink(path:*u8,from:i64,to:i64,start:i64,end:i64,requests:i64,fdout:i64,hash:*Sha256)->i64
3640func ch_usage_window_mode(path:*u8,from:i64,to:i64,start:i64,end:i64,requests:i64)->i64{return ch_usage_window_sink(path,from,to,start,end,requests,1,0 as *Sha256)}
3641func ch_usage_window(path:*u8,from:i64,to:i64,start:i64,end:i64)->i64
called by 3: mainmainmain calls 1: ch_usage_window_mode
3648func ch_usage_requests_file(path:*u8,from:i64,to:i64,start:i64,end:i64,output:*u8)->i64
3696func main(argc: i64, argv: *i64) -> i64
3878func chua_key(q:*u8,s:i64,n:i64,names:*u8)->i64
called by 1: chua_fields
3886func chua_fields(q:*u8,n:i64,names:*u8,count:i64,f:*i64,t:*NxJsonTok,r:*i64)->i64
3905func chua_lit(q:*u8,f:*i64,i:i64,value:*u8)->i64
3909func chua_num(q:*u8,f:*i64,i:i64)->i64
called by 1: chua_project calls 1: chu_uint
3912func chua_equal(a:*u8,f:*i64,i:i64,b:*u8,s:i64,n:i64)->i64
called by 1: chua_project
3916func chua_copy(q:*u8,f:*i64,i:i64,out:*u8,o:i64)->i64
called by 1: chua_project
3924func chua_project(q:*u8,n:i64,out:*u8,meta:*i64,t:*NxJsonTok,r:*i64,f:*i64)->i64
3968func chu_artifact_totals(raw:*u8,n:i64,from:i64,to:i64,table:*i64,t:*NxJsonTok,row:*i64,c:*i64,upstream:*u8)->i64