nx_token_counter.nx

nx_token_counter.nx -- SOVEREIGN LLM token estimation + budget-aware context packing: the "push our books into an LLM natively / build for RAG" foundation the library lacked (recon: BM25 + rerank exist, but no token counter and no context-packer). tok_estimate uses the well-known ~4-chars/token rule, FLOORED by a word count, as a deliberate slight OVER-estimate so budgeting never overflows the real context window. pack_text greedily keeps input lines IN ORDER until the token budget is hit, and emits an honest truncation marker if the document does not fully fit. HONEST: this is an ESTIMATE (no BPE table) for budgeting with a safety margin, not exact billing. license_tier: ORIGINAL