nx_textcut.nx

nx_textcut.nx -- LIB: TOKEN-BOUNDARY-SAFE TEXT CUTTING. The estate's ONE truncation ruler. THE DEFECT THIS EXISTS TO REMOVE, MEASURED LIVE ON nishifamily.com/search 2026-08-25: a renderer that truncates at a BYTE cap cuts mid-token. The stored doc "NIST Reference Fluid Thermodynamic and Transport Properties Database (REFPROP) Version 9" rendered a title ending "...Database (RE" (72-byte cap) and a snippet BEGINNING "FPROP) Version 9". TWO independent caps in one renderer (title 72B, sentence 320B) both cut mid-word. WHY IT SURVIVED: the damage is INVISIBLE AT THE CUT SITE. A title ending "(RE" looks merely truncated -- plausible. The mid-word START it creates surfaces in the NEXT span, which reads as a snippet bug, so every investigation looked at the snippet code and found nothing wrong with it. A CUT IS A CLAIM ABOUT WHERE A TOKEN ENDS; this lib makes that claim checkable and never silently wrong -- every cut lands on a real boundary or ANNOUNCES that it could not. ALSO CLOSES A DECLARED GAP ELSEWHERE: nx_search_snippet_extract.nx names both "UTF-8 codepoint awareness (no mid-codepoint truncation)" and "Word-boundary clipping" in its V2 SCOPE (TODO) and is unwired. Treating every byte >= 128 as a token byte closes BOTH at once: a multi-byte codepoint can never be split, because a cut inside one is a cut inside a token. NO THRESHOLDS. This lib is pure STRUCTURE (what a token is), not POLICY (how much to show), so it carries no conf and nothing here is tunable -- the cap is always the CALLER's. license_tier: ORIGINAL