`nx_memcliff.nx` source

↩ module page · 822 lines · 38401 B
1// nx_memcliff.nx -- SOVEREIGN owner of the MEMORY.md byte budget. Replaces memindex_enforce.py
2// (Stop hook) and the eviction half of memory_cliff_check.ps1 (SessionStart/PostToolUse) with ONE
3// nishi organ, so no interpreter sits in the hook path.
4//
5// WHY AN ORGAN AND NOT A SCRIPT
6// -----------------------------
7// MEMORY.md is injected into every session; past a byte limit the harness DROPS THE TAIL SILENTLY --
8// no error, no marker. The dropped lines are exactly the ones that would have told you they were
9// dropped. The rail-side enforcer cannot own this file (it runs NAS-side; MEMORY.md is laptop-local,
10// nx_memindex_put -> `no-md-file`), so ownership has to live here, next to the file.
11//
12// WHAT WAS ACTUALLY BROKEN -- MEASURED 2026-08-06, not assumed
13// ------------------------------------------------------------
14// The Python enforcer was NOT missing: it had run 64 times (64 eviction stamps in the overflow) and
15// had compacted the file at 13:47 the same day. It had JAMMED:
16//     "22013 bytes over budget but NO evictable entries found -- refusing to guess"
17// Root cause, measured: its rule NEVER EVICT THE NEWEST DAY pinned 8 of 8 candidates, because every
18// evictable entry carried today's date. Survivors after the pin: ZERO.
19// ★LAW: A PIN THAT CAN COVER 100% OF ITS CANDIDATES IS AN OFF SWITCH. The rule that protects today's
20// record disables enforcement precisely on the heaviest days -- the file grew 4.2KB in the two hours
21// after that successful compaction, with nothing left able to evict it.
22// The fix is not to drop the pin (it earned its place: a pure oldest-first cut once ate the current
23// session's own record) but to make it YIELD: keep the newest `keep_newest_day` entries of the newest
24// day and release the rest, rather than pinning the whole day unconditionally.
25//
26// SECOND MEASURED DEFECT -- the enforcer could not measure its subject in the unit the limit uses.
27// MEMORY.md is CRLF (63 CRLF, 0 bare LF). Python read it in text mode, which collapses CRLF to LF, so
28// it measured 22,013 against a file of 22,076 bytes -- UNDERSTATING by exactly the line count. The
29// harness cliff is a property of the bytes on disk, so every decision was made against a number that
30// was never the file's size.
31// ★LAW: A BUDGET ENFORCER THAT CANNOT MEASURE ITS SUBJECT IN THE UNIT THE LIMIT USES IS GUESSING.
32// ⚠WHAT I FIRST WROTE HERE AND THEN DISPROVED: I claimed the two owners also flipped the file's line
33// endings against each other (py writing LF, the ps1 writing CRLF). THE EXPERIMENT REFUTED IT -- a
34// CRLF fixture through the oracle came out CRLF=26 / bare-LF=0, because Python's text-mode WRITE
35// translates back on Windows, exactly undoing the read. The round trip is lossless; only the
36// measurement was wrong. Recorded rather than quietly deleted, because a plausible mechanism I had
37// not run is precisely the kind of premise that spends like evidence.
38// This organ works in RAW BYTES and reconstructs each line with its original terminator, so it is
39// byte-exact by construction rather than by a translation that happens to cancel out (verified: the
40// same CRLF fixture out at CRLF=26 / bare-LF=0).
41//
42// WHERE THE BUDGET NUMBERS COME FROM (and why they are config, per the no-magic-numbers rule)
43// -------------------------------------------------------------------------------------------
44// Three numbers were in use and all three were folklore: 17,100 (py budget), 24,400 (py hard),
45// 25,000 (ps1 cliff, itself back-derived from a recorded "26,229 B = 104.9%"). Attempts to recover the
46// true cliff from the 333 session transcripts FAILED and the attempt is recorded here so nobody
47// repeats it: the harness's injected memory context is never persisted to the transcript. What IS
48// measured, first-hand:
49//     22,076 B  loaded COMPLETE (2026-08-06: the file's on-disk last line was present in context)
50//     26,229 B  truncated       (2026-07-31 record)
51// So the cliff lies in (22,076, 26,229]. Defaults sit inside that bracket with real headroom, and a
52// 17,100-style target is REFUSED as a default: it would have evicted five of today's work records on
53// the strength of a number nobody measured.
54// ★LAW: AN UNMEASURED BUDGET IS NOT CONSERVATIVE JUST BECAUSE IT IS SMALL -- OVER-EVICTION DESTROYS
55// THE SAME INFORMATION THE CLIFF WOULD HAVE.
56//
57// SAFETY PROPERTIES (each inherited from a real incident)
58// ------------------------------------------------------
59// * FAIL-CLOSED ON UNREADABLE. Exists-but-unreadable -> refuse to write. (nx_ws_index_lib's 2026-07-31
60//   data-loss law: A GENERATOR THAT CANNOT READ THE FILE IT SPLICES INTO MUST REFUSE TO WRITE IT.
61//   ABSENT and UNREADABLE are opposite situations; only stat separates them.)
62// * PRESERVE BEFORE REMOVE. Victims are appended VERBATIM to the overflow file and the append is
63//   VERIFIED BY BYTE GROWTH before a single line leaves the index. Nothing is ever deleted.
64// * COMPARE-AND-SWAP. The file is re-measured immediately before install; if a concurrent writer
65//   touched it, ABORT without writing. (Hand compaction was measured losing this race inside ONE
66//   MINUTE.)
67// * ATOMIC INSTALL. tmp + renameat, so a reader sees old-whole or new-whole, never torn.
68// * NEVER TOUCHES PROTECTED REGIONS. The pinned head, and anything inside rail-owned COINDEX/CURATED
69//   marker blocks, is untouchable -- eviction only ever moves ordinary "- " entries carrying a topic
70//   link, so a moved line always leaves a followable pointer behind in the overflow.
71// * FIXED POINT. The eviction pointer line is UPDATED IN PLACE, never re-added, so a run near the
72//   budget cannot evict an entry to pay for its own bookkeeping.
73// * IDEMPOTENT + SILENT. Under target = zero writes, zero stdout, zero model tokens.
74//
75// usage:  nx_memcliff.elf [--report] [--dry-run]
76// exit:   0 = under target, or compacted;  1 = REFUSED (over target, could not act safely)
77// CONVERTED TO nx_memplane_lib 2026-08-07 (the last of the six). It keeps what is specific to the
78// budget -- the byte-exact line model, config parsing, date keys, the eviction ladder and the hook
79// JSON envelope -- and takes the generic byte/file primitives from the lib.
80// Sovereign: imports nx_memplane_lib. license_tier: ORIGINAL
81import "nx_memplane_lib.nx"
82
83const MC_DIR:   *u8 = "/mnt/c/Users/elder/.claude/projects/C--Users-elder/memory"
84const MC_N_MEM: *u8 = "MEMORY.md"
85const MC_N_OVF: *u8 = "reference-index-recent-refs-dormant.md"
86const MC_N_CNF: *u8 = "memcliff.conf"
87const MC_N_TMP: *u8 = "MEMORY.md.mctmp"
88
89const MC_MAXL:  i64 = 8192        // max lines in the index (64 today; vast headroom)
90const MC_BUF:   i64 = 4194304     // 4MB read buffer
91const MC_OUT:   i64 = 4194304     // 4MB assembled output
92const MC_MSG:   i64 = 8192        // report buffer
93
94// evidence-set defaults; every one overridable from memcliff.conf (no magic numbers in code)
95const MC_D_CLIFF:  i64 = 25000    // below the 26,229 B measured truncation
96const MC_D_TARGET: i64 = 23000    // evict down to this; above the 22,076 B measured-good load
97const MC_D_WARN:   i64 = 21000    // report headroom from here up
98const MC_D_KEEP:   i64 = 3        // newest-day entries always kept when the pin has to yield
99const MC_D_PIN:    i64 = 12       // pinned head: header + survival block, never evicted
100// THE SECOND CLIFF, measured 2026-08-06 and previously enforced by nobody. The harness applies a
101// LINE limit as well as a byte limit, whichever binds first. Probes at 100 B/line truncated at
102// EXACTLY 200 lines every time (20,000 B), while probes at 1000 B/line truncated at exactly 25,000 B
103// (24 lines). A file of few long lines is governed by bytes; a file of many short lines is governed
104// by LINES and can lose its tail with the byte budget entirely satisfied.
105const MC_D_MAXLINE: i64 = 200     // measured: line 201+ is not delivered
106const MC_D_TGTLINE: i64 = 180     // evict down to this, so ordinary appends don't immediately rebreach
107
108const MC_PTRTAG: *u8 = "verbatim by nx_memcliff:"
109
110// ---------- primitives -------------------------------------------------------------------------
111
112// mc_len / mc_streq / mc_cat / mc_catn / mc_write_all / mc_say / mc_read / mc_join deleted at the
113// lib conversion -- they were byte-identical copies of the lib's primitives in all six organs.
114
115// does NUL-terminated `pat` occur at buf[pos]? (bounded by `lim`)
116func mc_at(buf: *u8, pos: i64, lim: i64, pat: *u8) -> i64 {
  let pl: i64 = mp_len(pat)
  if pos + pl > lim { return 0 }
  var j: i64 = 0
  var ok: i64 = 1
  while j < pl {
      if buf[pos + j] != pat[j] { ok = 0; j = pl } else { j = j + 1 }
  }
  return ok
125}
126
127// first index of `pat` in buf[from..lim), or -1
128func mc_find(buf: *u8, from: i64, lim: i64, pat: *u8) -> i64 {
  var i: i64 = from
  var r: i64 = 0 - 1
  var go: i64 = 1
  while go == 1 {
      if i >= lim { go = 0 } else {
          if mc_at(buf, i, lim, pat) == 1 { r = i; go = 0 } else { i = i + 1 }
      }
  }
  return r
138}
139
140func mc_isdigit(c: u8) -> i64 {
  if c < (48 as u8) { return 0 }
  if c > (57 as u8) { return 0 }
  return 1
144}
145
146
147// Emit a report. `json` = 0 -> raw stdout (the Stop hook shows it verbatim). `json` = 1 -> the hook
148// envelope, so SessionStart/PostToolUse can carry the text into context WITHOUT a PowerShell wrapper
149// doing the JSON assembly. Keeping this here is the point: no interpreter in the hook path.
150func mc_emit(json: i64, buf: *u8, n: i64) -> i64 {
  if json == 0 { return mp_write_all(1, buf, n) }
  let j: *u8 = sys_mmap(MC_MSG * 4)
  var o: i64 = 0
  o = mp_cat(j, o, "{\"systemMessage\":\"" as *u8)
  var pass: i64 = 0
  while pass < 2 {
      var i: i64 = 0
      while i < n {
          let c: u8 = buf[i]
          if c == (34 as u8) { j[o] = 92 as u8; o = o + 1; j[o] = 34 as u8; o = o + 1 } else {
              if c == (92 as u8) { j[o] = 92 as u8; o = o + 1; j[o] = 92 as u8; o = o + 1 } else {
                  if c == (10 as u8) { j[o] = 92 as u8; o = o + 1; j[o] = 110 as u8; o = o + 1 } else {
                      if c == (13 as u8) { o = o + 0 } else { j[o] = c; o = o + 1 }
                  }
              }
          }
          i = i + 1
      }
      if pass == 0 {
          o = mp_cat(j, o, "\",\"hookSpecificOutput\":{\"hookEventName\":\"SessionStart\",\"additionalContext\":\"" as *u8)
      }
      pass = pass + 1
  }
  o = mp_cat(j, o, "\"}}" as *u8)
  return mp_write_all(1, j, o)
176}
177
178// dir + "/" + name -> a fresh NUL-terminated buffer. Lets --dir point the organ at a FIXTURE, which
179// is what makes the differential test against the known-good oracle possible at all.
180func mc_join(dir: *u8, name: *u8) -> *u8 {
  let dst: *u8 = sys_mmap(4096)
  var o: i64 = 0
  var i: i64 = 0
  while dir[i] != (0 as u8) { dst[o] = dir[i]; o = o + 1; i = i + 1 }
  if o > 0 { if dst[o - 1] != (47 as u8) { dst[o] = 47 as u8; o = o + 1 } }
  i = 0
  while name[i] != (0 as u8) { dst[o] = name[i]; o = o + 1; i = i + 1 }
  dst[o] = 0 as u8
  return dst
190}
191
192// TRUE byte size via lseek(END) -- deliberately NOT struct-stat offset arithmetic, which would be an
193// assumption about layout. -1 if the file cannot be opened.
194func mc_fsize(path: *u8) -> i64 {
  let fd: i64 = sys_openat_rd(path)
  if fd < 0 { return 0 - 1 }
  let n: i64 = sys_lseek(fd, 0, 2)
  sys_close(fd)
  return n
200}
201
202func mc_exists(path: *u8) -> i64 {
  let stb: *u8 = sys_mmap(160)
  if sys_fstatat(path, stb) == 0 { return 1 }
  return 0
206}
207
208// whole-file read into buf; returns length, or -1 if open failed
209func mp_readf(path: *u8, buf: *u8, cap: i64) -> i64 {
  let fd: i64 = sys_openat_rd(path)
  if fd < 0 { return 0 - 1 }
  var total: i64 = 0
  var go: i64 = 1
  while go == 1 {
      let n: i64 = sys_read(fd, ((buf as i64) + total) as *u8, cap - total)
      if n <= 0 { go = 0 } else {
          total = total + n
          if total >= cap { go = 0 }
      }
  }
  sys_close(fd)
  return total
223}
224
225// ---------- config -----------------------------------------------------------------------------
226
227// read `key=<int>` from the conf buffer; `dflt` when absent or malformed.
228func mc_conf(cbuf: *u8, clen: i64, key: *u8, dflt: i64) -> i64 {
  if clen <= 0 { return dflt }
  var i: i64 = 0
  var val: i64 = dflt
  var found: i64 = 0
  while i < clen {
      var atline: i64 = 0
      if i == 0 { atline = 1 }
      if i > 0 { if cbuf[i - 1] == (10 as u8) { atline = 1 } }
      if atline == 1 {
          if found == 0 {
              if mc_at(cbuf, i, clen, key) == 1 {
                  var p: i64 = i + mp_len(key)
                  if p < clen {
                      if cbuf[p] == (61 as u8) {          // '='
                          p = p + 1
                          var acc: i64 = 0
                          var any: i64 = 0
                          var go: i64 = 1
                          while go == 1 {
                              if p >= clen { go = 0 } else {
                                  if mc_isdigit(cbuf[p]) == 1 {
                                      acc = acc * 10 + ((cbuf[p] as i64) - 48)
                                      any = 1
                                      p = p + 1
                                  } else { go = 0 }
                              }
                          }
                          if any == 1 { val = acc; found = 1 }
                      }
                  }
              }
          }
      }
      i = i + 1
  }
  return val
265}
266
267// ---------- line model -------------------------------------------------------------------------
268// ls[i] = start offset, ll[i] = length INCLUDING the line terminator. Reconstructing from these
269// preserves CRLF byte-exactly -- the organ can never flip a file's line endings.
270
271func mc_split(buf: *u8, n: i64, ls: *i64, ll: *i64, cap: i64) -> i64 {
  var cnt: i64 = 0
  var start: i64 = 0
  var i: i64 = 0
  while i < n {
      if buf[i] == (10 as u8) {
          if cnt < cap { ls[cnt] = start; ll[cnt] = i - start + 1; cnt = cnt + 1 }
          start = i + 1
      }
      i = i + 1
  }
  if start < n {
      if cnt < cap { ls[cnt] = start; ll[cnt] = n - start; cnt = cnt + 1 }
  }
  return cnt
286}
287
288// date key for a line: YYYY-MM-DD -> YYYYMMDD; else MM-DD -> 2026MMDD; else 99999999 (undated,
289// sorts last = kept longest, matching the known-good behaviour).
290func mc_datekey(buf: *u8, s: i64, e: i64) -> i64 {
  var i: i64 = s
  // full YYYY-MM-DD
  while i + 10 <= e {
      var ok: i64 = 1
      if mc_isdigit(buf[i]) == 0 { ok = 0 }
      if ok == 1 { if mc_isdigit(buf[i + 1]) == 0 { ok = 0 } }
      if ok == 1 { if mc_isdigit(buf[i + 2]) == 0 { ok = 0 } }
      if ok == 1 { if mc_isdigit(buf[i + 3]) == 0 { ok = 0 } }
      if ok == 1 { if buf[i + 4] != (45 as u8) { ok = 0 } }
      if ok == 1 { if mc_isdigit(buf[i + 5]) == 0 { ok = 0 } }
      if ok == 1 { if mc_isdigit(buf[i + 6]) == 0 { ok = 0 } }
      if ok == 1 { if buf[i + 7] != (45 as u8) { ok = 0 } }
      if ok == 1 { if mc_isdigit(buf[i + 8]) == 0 { ok = 0 } }
      if ok == 1 { if mc_isdigit(buf[i + 9]) == 0 { ok = 0 } }
      if ok == 1 {
          var v: i64 = 0
          v = v * 10 + ((buf[i] as i64) - 48)
          v = v * 10 + ((buf[i + 1] as i64) - 48)
          v = v * 10 + ((buf[i + 2] as i64) - 48)
          v = v * 10 + ((buf[i + 3] as i64) - 48)
          v = v * 10 + ((buf[i + 5] as i64) - 48)
          v = v * 10 + ((buf[i + 6] as i64) - 48)
          v = v * 10 + ((buf[i + 8] as i64) - 48)
          v = v * 10 + ((buf[i + 9] as i64) - 48)
          return v
      }
      i = i + 1
  }
  // bare MM-DD
  i = s
  while i + 5 <= e {
      var ok2: i64 = 1
      if mc_isdigit(buf[i]) == 0 { ok2 = 0 }
      if ok2 == 1 { if mc_isdigit(buf[i + 1]) == 0 { ok2 = 0 } }
      if ok2 == 1 { if buf[i + 2] != (45 as u8) { ok2 = 0 } }
      if ok2 == 1 { if mc_isdigit(buf[i + 3]) == 0 { ok2 = 0 } }
      if ok2 == 1 { if mc_isdigit(buf[i + 4]) == 0 { ok2 = 0 } }
      // must not be part of a longer digit run (guards against slicing a YYYY-MM-DD tail)
      if ok2 == 1 { if i > s { if mc_isdigit(buf[i - 1]) == 1 { ok2 = 0 } } }
      if ok2 == 1 { if i + 5 < e { if mc_isdigit(buf[i + 5]) == 1 { ok2 = 0 } } }
      if ok2 == 1 {
          var v2: i64 = 2026
          v2 = v2 * 10 + ((buf[i] as i64) - 48)
          v2 = v2 * 10 + ((buf[i + 1] as i64) - 48)
          v2 = v2 * 10 + ((buf[i + 3] as i64) - 48)
          v2 = v2 * 10 + ((buf[i + 4] as i64) - 48)
          return v2
      }
      i = i + 1
  }
  return 99999999
342}
343
344// a line is EVICTABLE only if it is an ordinary entry carrying a topic link, so a moved line always
345// leaves a followable pointer behind.
346func mc_is_entry(buf: *u8, s: i64, e: i64) -> i64 {
  if e - s < 4 { return 0 }
  if buf[s] != (45 as u8) { return 0 }        // '-'
  if buf[s + 1] != (32 as u8) { return 0 }    // ' '
  if mc_find(buf, s, e, "](" as *u8) < 0 { return 0 }
  if mc_find(buf, s, e, ".md)" as *u8) < 0 { return 0 }
  return 1
353}
354
355// ---------- main -------------------------------------------------------------------------------
356
357func main(argc: i64, argv: *i64) -> i64 {
  var dry: i64 = 0
  var report: i64 = 0
  var jsonf: i64 = 0
  var announce: i64 = 0
  var dir: *u8 = MC_DIR
  var ai: i64 = 1
  while ai < argc {
      let a: *u8 = argv[ai] as *u8
      if mp_streq(a, "--dry-run" as *u8) == 1 { dry = 1 }
      if mp_streq(a, "--report" as *u8) == 1 { report = 1 }
      if mp_streq(a, "--json" as *u8) == 1 { jsonf = 1 }
      if mp_streq(a, "--announce" as *u8) == 1 { announce = 1; report = 1 }
      if mp_streq(a, "--dir" as *u8) == 1 {
          if ai + 1 < argc { dir = argv[ai + 1] as *u8; ai = ai + 1 }
      }
      ai = ai + 1
  }
375
  let MC_MEM:  *u8 = mc_join(dir, MC_N_MEM)
  let MC_OVF:  *u8 = mc_join(dir, MC_N_OVF)
  let MC_CONF: *u8 = mc_join(dir, MC_N_CNF)
  let MC_TMP:  *u8 = mc_join(dir, MC_N_TMP)
380
  let msg: *u8 = sys_mmap(MC_MSG)
382
  // ---- config (data-driven thresholds) ----
  let cbuf: *u8 = sys_mmap(65536)
  let clen: i64 = mp_readf(MC_CONF, cbuf, 65536)
  let cliff:  i64 = mc_conf(cbuf, clen, "cliff_bytes" as *u8,     MC_D_CLIFF)
  let target: i64 = mc_conf(cbuf, clen, "target_bytes" as *u8,    MC_D_TARGET)
  let warn:   i64 = mc_conf(cbuf, clen, "warn_bytes" as *u8,      MC_D_WARN)
  let keepn:  i64 = mc_conf(cbuf, clen, "keep_newest_day" as *u8, MC_D_KEEP)
  let pinl:   i64 = mc_conf(cbuf, clen, "pin_lines" as *u8,       MC_D_PIN)
  let maxl:   i64 = mc_conf(cbuf, clen, "max_lines" as *u8,       MC_D_MAXLINE)
  let tlines: i64 = mc_conf(cbuf, clen, "target_lines" as *u8,    MC_D_TGTLINE)
393
  // ---- FAIL-CLOSED read ----
  let size0: i64 = mc_fsize(MC_MEM)
  if size0 < 0 {
      if mc_exists(MC_MEM) == 1 {
          var m: i64 = mp_cat(msg, 0, "nx_memcliff: REFUSED -- MEMORY.md EXISTS but could not be read. Refusing to write a file I cannot see.\n" as *u8)
          mc_emit(jsonf, msg, m)
          return 1
      }
      return 0                                    // genuinely absent: nothing to enforce
  }
404
  // Read BEFORE deciding anything: the line cliff is a property of the content, not the size, so
  // an enforcer that short-circuits on bytes alone can never see the axis that is about to bite.
  let buf: *u8 = sys_mmap(MC_BUF)
  let n: i64 = mp_readf(MC_MEM, buf, MC_BUF)
  if n <= 0 {
      var m2: i64 = mp_cat(msg, 0, "nx_memcliff: REFUSED -- MEMORY.md unreadable at eviction time.\n" as *u8)
      mc_emit(jsonf, msg, m2)
      return 1
  }
414
  let ls: *i64 = sys_mmap(8 * MC_MAXL) as *i64
  let ll: *i64 = sys_mmap(8 * MC_MAXL) as *i64
  let lc: i64 = mc_split(buf, n, ls, ll, MC_MAXL)
418
  // ---- report mode: state the headroom on BOTH axes, never write ----
  if report == 1 {
      var loud: i64 = 0
      if size0 >= warn { loud = 1 }
      if lc >= tlines { loud = 1 }
      // ★ DISCOVERABILITY IS PART OF THE CAPABILITY. Operator, 2026-08-06: a tool that needs "greps
      // and all this other bullshit" to find at session start is not SOTA. The memory plane is
      // laptop-local, and the MCP transport is remote http (nishifamily.com/mcp) -- every MCP tool
      // executes NAS-side and cannot read this directory, which is the same no-md-file wall the rail
      // hits. So MCP registration cannot carry these organs; the SessionStart surface can, and it is
      // already proven and already reads this file. --announce rides the existing call: live budget
      // state AND the organ roster, one invocation, no extra cost.
      if announce == 1 { loud = 1 }
      if loud == 1 {
          var r: i64 = mp_cat(msg, 0, "MEMORY.md " as *u8)
          r = mp_catn(msg, r, size0)
          r = mp_cat(msg, r, " B of the " as *u8)
          r = mp_catn(msg, r, cliff)
          r = mp_cat(msg, r, " B cliff (" as *u8)
          r = mp_catn(msg, r, cliff - size0)
          r = mp_cat(msg, r, " B headroom), " as *u8)
          r = mp_catn(msg, r, lc)
          r = mp_cat(msg, r, " of " as *u8)
          r = mp_catn(msg, r, maxl)
          r = mp_cat(msg, r, " lines (" as *u8)
          r = mp_catn(msg, r, maxl - lc)
          r = mp_cat(msg, r, " spare). Targets " as *u8)
          r = mp_catn(msg, r, target)
          r = mp_cat(msg, r, " B / " as *u8)
          r = mp_catn(msg, r, tlines)
          r = mp_cat(msg, r, " lines.\n" as *u8)
          if announce == 1 {
              r = mp_cat(msg, r, "MEMORY-PLANE ORGANS (laptop-local; MCP is remote http so it cannot reach this dir). Run via wsl -e /mnt/c/Users/elder/nishi-core/nxc2/_offc/<organ>:\n" as *u8)
              r = mp_cat(msg, r, "  nx_memfind --file <f> | <terms>  BM25 ranked retrieval -- ASK THIS BEFORE WRITING A NEW MEMORY\n" as *u8)
              r = mp_cat(msg, r, "  nx_memroot [--apply]             from-god rooting: node->parent->ORIGIN, censuses FLOATING\n" as *u8)
              r = mp_cat(msg, r, "  nx_memhealth [--dups]            dangling / orphans / duplicate candidates (read-only)\n" as *u8)
              r = mp_cat(msg, r, "  nx_memfix [--apply]              repairs ONLY provably-safe dead links\n" as *u8)
              r = mp_cat(msg, r, "  nx_memorph --apply               regenerates the orphan catalogue\n" as *u8)
          }
          mc_emit(jsonf, msg, r)
      }
      return 0
  }
462
  // ---- IDEMPOTENT: under BOTH targets = zero writes, zero stdout ----
  var over: i64 = 0
  if size0 > target { over = 1 }
  if lc > tlines { over = 1 }
  if over == 0 { return 0 }
468
  // ---- classify ----
  let cand: *i64 = sys_mmap(8 * MC_MAXL) as *i64
  let dkey: *i64 = sys_mmap(8 * MC_MAXL) as *i64
  var inrail: i64 = 0
  var i: i64 = 0
  while i < lc {
      let s: i64 = ls[i]
      let e: i64 = ls[i] + ll[i]
      cand[i] = 0
      dkey[i] = 99999999
      if mc_find(buf, s, e, "COINDEX:BEGIN" as *u8) >= 0 { inrail = 1 }
      if mc_find(buf, s, e, "CURATED:BEGIN" as *u8) >= 0 { inrail = 1 }
      var protd: i64 = 0
      if i < pinl { protd = 1 }
      if inrail == 1 { protd = 1 }
      if protd == 0 {
          if mc_is_entry(buf, s, e) == 1 {
              cand[i] = 1
              dkey[i] = mc_datekey(buf, s, e)
          }
      }
      if mc_find(buf, s, e, "COINDEX:END" as *u8) >= 0 { inrail = 0 }
      if mc_find(buf, s, e, "CURATED:END" as *u8) >= 0 { inrail = 0 }
      i = i + 1
  }
494
  // ---- the newest-day pin, AND the yield that stops it becoming an off switch ----
  var newest: i64 = 0 - 1
  i = 0
  while i < lc {
      if cand[i] == 1 {
          if dkey[i] != 99999999 { if dkey[i] > newest { newest = dkey[i] } }
      }
      i = i + 1
  }
504
  var pool: i64 = 0                                   // candidates NOT on the newest day
  i = 0
  while i < lc {
      if cand[i] == 1 { if dkey[i] != newest { pool = pool + 1 } }
      i = i + 1
  }
511
  // eligible[] = may be evicted this run
  let elig: *i64 = sys_mmap(8 * MC_MAXL) as *i64
  i = 0
  while i < lc { elig[i] = 0; i = i + 1 }
  i = 0
  while i < lc {
      if cand[i] == 1 { if dkey[i] != newest { elig[i] = 1 } }
      i = i + 1
  }
521
  var jam: i64 = 0
523
  // The pointer line is part of the OUTPUT, so it must be part of the ARITHMETIC. Detect the prior
  // pointer and its cost BEFORE choosing victims: a loop that ignores it lands over target by
  // exactly the pointer's length -- measured 23,043 B against a 23,000 B target, 43 B over, which
  // is precisely where the margin matters. Worst-case digit count, so the estimate can only ever
  // over-evict by a few bytes, never under.
  var prior: i64 = 0
  var ptrline: i64 = 0 - 1
  i = 0
  while i < lc {
      let pf: i64 = mc_find(buf, ls[i], ls[i] + ll[i], MC_PTRTAG)
      if pf >= 0 {
          if ptrline < 0 {
              ptrline = i
              var q: i64 = pf + mp_len(MC_PTRTAG)
              var acc: i64 = 0
              var any: i64 = 0
              var g2: i64 = 1
              while g2 == 1 {
                  if q >= ls[i] + ll[i] { g2 = 0 } else {
                      if mc_isdigit(buf[q]) == 1 { acc = acc * 10 + ((buf[q] as i64) - 48); any = 1; q = q + 1 } else {
                          if any == 1 { g2 = 0 } else { q = q + 1 }
                      }
                  }
              }
              if any == 1 { prior = acc }
          }
      }
      i = i + 1
  }
  let scratch: *u8 = sys_mmap(1024)
  var pc: i64 = mp_cat(scratch, 0, "- EVICTED -> [OVERFLOW](reference-index-recent-refs-dormant.md) " as *u8)
  pc = mp_cat(scratch, pc, MC_PTRTAG)
  pc = mp_cat(scratch, pc, " 999999 entries." as *u8)
  let ptr_add: i64 = pc + 2                       // + CRLF, the worst case
  var ptr_del: i64 = 0
  var line_delta: i64 = 1
  if ptrline >= 0 { ptr_del = ll[ptrline]; line_delta = 0 }
561
  // ---- choose victims: oldest key first; within a key, bottom-most first ----
  let vic: *i64 = sys_mmap(8 * MC_MAXL) as *i64
  var nvic: i64 = 0
  var running: i64 = size0
566
  // TWO PHASES, because the newest-day pin is an off switch at ANY coverage, not only at 100%.
  // The first fix here yielded when the non-newest-day pool was EMPTY. Measured on the live file
  // 2026-08-06: the pool was ONE -- 13 of 14 candidates carried today's date -- so the organ evicted
  // a single line, saved nothing, and refused while the file sat 1,797 B OVER the measured cliff.
  // ★★★★★★ A PIN DOES NOT HAVE TO COVER EVERYTHING TO DISABLE ENFORCEMENT; IT ONLY HAS TO COVER
  // ENOUGH THAT WHAT REMAINS CANNOT CLOSE THE GAP. The trigger is therefore not "is the pool empty"
  // but "did the pool actually reach the target" -- phase 0 spends the unpinned candidates, and if
  // that was not enough, phase 1 releases the newest day except the `keepn` nearest the head.
  var phase: i64 = 0
  while phase < 2 {
  var go: i64 = 1
  while go == 1 {
      // drive on BOTH cliffs: stop only when the file is under the byte target AND the line target
      var more: i64 = 0
      if running - ptr_del + ptr_add > target { more = 1 }
      if lc - nvic + line_delta > tlines { more = 1 }
      if more == 0 { go = 0 } else {
          var best: i64 = 0 - 1
          var bestk: i64 = 0
          i = 0
          while i < lc {
              if elig[i] == 1 {
                  var take: i64 = 0
                  if best < 0 { take = 1 } else {
                      if dkey[i] < bestk { take = 1 }
                      if dkey[i] == bestk { take = 1 }      // later index wins => bottom-most first
                  }
                  if take == 1 { best = i; bestk = dkey[i] }
              }
              i = i + 1
          }
          if best < 0 { go = 0 } else {
              elig[best] = 0
              vic[nvic] = best
              nvic = nvic + 1
              running = running - ll[best]
          }
      }
  }
      // THE YIELD, driven by outcome rather than by pool size: only if spending everything
      // unpinned still left the file over target does the newest day get released.
      var still: i64 = 0
      if running - ptr_del + ptr_add > target { still = 1 }
      if lc - nvic + line_delta > tlines { still = 1 }
      if still == 1 {
          if phase == 0 {
              jam = 1
              // ★★★★★★ A FLOOR SET AGAINST A LARGE CANDIDATE POOL BECOMES A TOTAL BLOCK ONCE THE
              // POOL SHRINKS TO THE FLOOR. Measured live 2026-08-07: the index held FOUR eligible
              // candidates -- one undated, three newest-day -- so phase 0 spent the single unpinned
              // one and phase 1 released NOTHING, because keep_newest_day=3 protected exactly the
              // three that remained. The organ was 298 B from the cliff and structurally unable to
              // act. Same shape as the pin bug this organ already fixed once: the guard became the
              // off switch when the pool shrank to its own size.
              // The floor exists to stop ROUTINE target-driven eviction eating today's record. It
              // must never stop the organ preventing SILENT TRUNCATION, because eviction preserves
              // the line VERBATIM in the overflow while the cliff destroys it unseen.
              // ⇒ over the cliff, the floor drops to 1. Never to 0: something of today always stays.
              var effkeep: i64 = keepn
              if size0 > cliff { effkeep = 1 }
              var kept: i64 = 0
              i = 0
              while i < lc {
                  if cand[i] == 1 {
                      if dkey[i] == newest {
                          if kept < effkeep { kept = kept + 1 } else { elig[i] = 1 }
                      }
                  }
                  i = i + 1
              }
          }
      }
      phase = phase + 1
  }
641
  if nvic == 0 {
      var m3: i64 = mp_cat(msg, 0, "*** nx_memcliff: MEMORY.md is " as *u8)
      m3 = mp_catn(msg, m3, size0)
      m3 = mp_cat(msg, m3, " B, over the " as *u8)
      m3 = mp_catn(msg, m3, target)
      m3 = mp_cat(msg, m3, " B target, and NOTHING IS ELIGIBLE TO EVICT -- every remaining entry is protected (pinned head or rail-owned block). A human must compact. ***\n" as *u8)
      mc_emit(jsonf, msg, m3)
      return 1
  }
651
  // ---- PRESERVE FIRST: append victims verbatim, then VERIFY BY BYTE GROWTH ----
  let ovf0: i64 = mc_fsize(MC_OVF)
  let ob: *u8 = sys_mmap(MC_OUT)
  var oo: i64 = 0
  oo = mp_cat(ob, oo, "\n\n## EVICTED FROM MEMORY.md by nx_memcliff (verbatim, nothing dropped) -- index was " as *u8)
  oo = mp_catn(ob, oo, size0)
  oo = mp_cat(ob, oo, " B against a " as *u8)
  oo = mp_catn(ob, oo, target)
  oo = mp_cat(ob, oo, " B target.\n" as *u8)
  var k: i64 = nvic - 1
  while k >= 0 {
      let vi: i64 = vic[k]
      var c: i64 = 0
      while c < ll[vi] { ob[oo] = buf[ls[vi] + c]; oo = oo + 1; c = c + 1 }
      if ob[oo - 1] != (10 as u8) { ob[oo] = 10 as u8; oo = oo + 1 }
      k = k - 1
  }
  if dry == 0 {
      let ofd: i64 = sys_openat_append(MC_OVF, 420)
      if ofd < 0 {
          var m4: i64 = mp_cat(msg, 0, "*** nx_memcliff: REFUSED -- over target but the overflow file could not be opened. NOTHING EVICTED (refusing to drop lines I cannot preserve). ***\n" as *u8)
          mc_emit(jsonf, msg, m4)
          return 1
      }
      mp_write_all(ofd, ob, oo)
      sys_close(ofd)
      let ovf1: i64 = mc_fsize(MC_OVF)
      if ovf1 - ovf0 < oo {
          var m5: i64 = mp_cat(msg, 0, "*** nx_memcliff: REFUSED -- the overflow append did not land (grew " as *u8)
          m5 = mp_catn(msg, m5, ovf1 - ovf0)
          m5 = mp_cat(msg, m5, " B, expected " as *u8)
          m5 = mp_catn(msg, m5, oo)
          m5 = mp_cat(msg, m5, " B). NOTHING REMOVED. ***\n" as *u8)
          mc_emit(jsonf, msg, m5)
          return 1
      }
  }
689
  // ---- assemble the survivor file, preserving every original terminator byte-exactly ----
  // the eviction pointer is a FIXED POINT: the prior one was located and its count absorbed above
  // (before victim selection, so its byte cost is inside the arithmetic); here it is simply dropped
  // and ONE updated pointer re-emitted at the position of the first victim.
  var firstv: i64 = lc
  k = 0
  while k < nvic { if vic[k] < firstv { firstv = vic[k] } k = k + 1 }
  if firstv < pinl { firstv = pinl }
698
  // CRLF or LF, taken from the file itself -- never assumed
  var crlf: i64 = 0
  if lc > 0 { if ll[0] >= 2 { if buf[ls[0] + ll[0] - 2] == (13 as u8) { crlf = 1 } } }
702
  let out: *u8 = sys_mmap(MC_OUT)
  var o: i64 = 0
  i = 0
  while i < lc {
      if i == firstv {
          o = mp_cat(out, o, "- EVICTED -> [OVERFLOW](reference-index-recent-refs-dormant.md) " as *u8)
          o = mp_cat(out, o, MC_PTRTAG)
          o = mp_cat(out, o, " " as *u8)
          o = mp_catn(out, o, prior + nvic)
          o = mp_cat(out, o, " entries." as *u8)
          if crlf == 1 { out[o] = 13 as u8; o = o + 1 }
          out[o] = 10 as u8; o = o + 1
      }
      var drop: i64 = 0
      if i == ptrline { drop = 1 }
      k = 0
      while k < nvic { if vic[k] == i { drop = 1 } k = k + 1 }
      if drop == 0 {
          var c2: i64 = 0
          while c2 < ll[i] { out[o] = buf[ls[i] + c2]; o = o + 1; c2 = c2 + 1 }
      }
      i = i + 1
  }
726
  // ---- REFUSE A NO-OP: a run that does not shrink the file is pure churn ----
  // ⚠A REFUSAL THAT DOES NOT SHOW ITS ARITHMETIC CANNOT BE DEBUGGED. This branch fired while the
  // live file sat 1,797 B OVER the measured cliff, and the message gave nothing to reason from.
  // It also must NOT be silent-and-zero when the file is over the CLIFF (not merely over target):
  // that is a fail-open in exactly the situation the organ exists for.
  if o >= size0 {
      var m6: i64 = mp_cat(msg, 0, "nx_memcliff: no net saving (in " as *u8)
      m6 = mp_catn(msg, m6, size0)
      m6 = mp_cat(msg, m6, " B, out " as *u8)
      m6 = mp_catn(msg, m6, o)
      m6 = mp_cat(msg, m6, " B, lines " as *u8)
      m6 = mp_catn(msg, m6, lc)
      m6 = mp_cat(msg, m6, ", victims " as *u8)
      m6 = mp_catn(msg, m6, nvic)
      m6 = mp_cat(msg, m6, ", ptr +" as *u8)
      m6 = mp_catn(msg, m6, ptr_add)
      m6 = mp_cat(msg, m6, "/-" as *u8)
      m6 = mp_catn(msg, m6, ptr_del)
      m6 = mp_cat(msg, m6, ") -- refusing to churn." as *u8)
      if size0 > cliff {
          m6 = mp_cat(msg, m6, "\n*** AND THE FILE IS OVER THE MEASURED CLIFF BY " as *u8)
          m6 = mp_catn(msg, m6, size0 - cliff)
          m6 = mp_cat(msg, m6, " B -- THE TAIL IS BEING DROPPED FROM EVERY SESSION RIGHT NOW AND I CANNOT FIX IT. A human must compact. ***" as *u8)
      }
      m6 = mp_cat(msg, m6, "\n" as *u8)
      mc_emit(jsonf, msg, m6)
      if size0 > cliff { return 1 }
      return 0
  }
756
  if dry == 1 {
      var m7: i64 = mp_cat(msg, 0, "nx_memcliff: DRY-RUN " as *u8)
      m7 = mp_catn(msg, m7, size0)
      m7 = mp_cat(msg, m7, " -> " as *u8)
      m7 = mp_catn(msg, m7, o)
      m7 = mp_cat(msg, m7, " B, would evict " as *u8)
      m7 = mp_catn(msg, m7, nvic)
      m7 = mp_cat(msg, m7, " entries" as *u8)
      if jam == 1 { m7 = mp_cat(msg, m7, " (NEWEST-DAY PIN YIELDED: every candidate was newest-day; kept the " as *u8); m7 = mp_catn(msg, m7, keepn); m7 = mp_cat(msg, m7, " nearest the head)" as *u8) }
      m7 = mp_cat(msg, m7, ".\n" as *u8)
      mc_emit(jsonf, msg, m7)
      return 0
  }
770
  // ---- COMPARE-AND-SWAP: abort if a concurrent writer touched the file since we read it ----
  if mc_fsize(MC_MEM) != size0 {
      var m8: i64 = mp_cat(msg, 0, "nx_memcliff: MEMORY.md changed under us mid-eviction (concurrent writer) -- ABORTED, nothing removed. The entries were already preserved to the overflow; the next run retries.\n" as *u8)
      mc_emit(jsonf, msg, m8)
      return 1
  }
777
  // ---- ATOMIC INSTALL ----
  let wfd: i64 = sys_openat_wr(MC_TMP, 420)
  if wfd < 0 {
      var m9: i64 = mp_cat(msg, 0, "*** nx_memcliff: REFUSED -- could not open the temp file; nothing removed. ***\n" as *u8)
      mc_emit(jsonf, msg, m9)
      return 1
  }
  let wrote: i64 = mp_write_all(wfd, out, o)
  sys_close(wfd)
  if wrote != o {
      var ma: i64 = mp_cat(msg, 0, "*** nx_memcliff: REFUSED -- short write to the temp file; live file untouched. ***\n" as *u8)
      mc_emit(jsonf, msg, ma)
      return 1
  }
  if sys_renameat(MC_TMP, MC_MEM) != 0 {
      var mb: i64 = mp_cat(msg, 0, "*** nx_memcliff: REFUSED -- atomic rename failed; live file untouched. ***\n" as *u8)
      mc_emit(jsonf, msg, mb)
      return 1
  }
797
  var m: i64 = mp_cat(msg, 0, "nx_memcliff: MEMORY.md " as *u8)
  m = mp_catn(msg, m, size0)
  m = mp_cat(msg, m, " -> " as *u8)
  m = mp_catn(msg, m, o)
  m = mp_cat(msg, m, " B (target " as *u8)
  m = mp_catn(msg, m, target)
  m = mp_cat(msg, m, ", cliff " as *u8)
  m = mp_catn(msg, m, cliff)
  m = mp_cat(msg, m, "); evicted " as *u8)
  m = mp_catn(msg, m, nvic)
  m = mp_cat(msg, m, " entries VERBATIM to the overflow (nothing deleted); lines " as *u8)
  m = mp_catn(msg, m, lc)
  m = mp_cat(msg, m, " -> " as *u8)
  m = mp_catn(msg, m, lc - nvic)
  m = mp_cat(msg, m, " of max " as *u8)
  m = mp_catn(msg, m, maxl)
  if jam == 1 {
      m = mp_cat(msg, m, ". NOTE: the unpinned candidates could not reach the target, so the newest-day pin YIELDED and kept the " as *u8)
      m = mp_catn(msg, m, keepn)
      m = mp_cat(msg, m, " nearest the head -- without this the enforcer would have refused and the budget would be unenforced" as *u8)
  }
  m = mp_cat(msg, m, ".\n" as *u8)
  mc_emit(jsonf, msg, m)
  return 0
822}
nx_memcliff.nx source

`nx_memcliff.nx` source
