code wiki / (root) / nx_chat_warmcache.nx

nx_chat_warmcache.nx

buildroot/runtime/nx_chat_warmcache.nx

8620 B173 linesdepth 13pulls 59 transitivereach 0 importersview sourcekind tool
docsdependenciesstructsconstsfunctions

about

nx_chat_warmcache.nx -- prefix-cache win WIRED into the LIVE engine (serving census Q3, end-to-end). A multi-turn companion chat on ONE persistent KV cache: the persona system prompt + conversation history are prefilled ONCE and their KV is REUSED every subsequent turn -- turn N only prefills its NEW user tokens, not the whole prompt. This is nx_prefix_cache's measured 85% win proven on the REAL forward (nx_f32_llm_forward_v4 + a warm nx_f32_kv_cache), not a simulation. Naive serving resets the cache each turn and re-prefills everything; this keeps it warm. license_tier: ORIGINAL expect_exit: 0

dependencies 18 imports · 0 importers

nx_syscalls.nx nx_tier.nx nx_bpe.nx nx_gguf.nx nx_gguf_load.nx nx_gguf_meta.nx nx_f32.nx nx_f32_kv_cache.nx nx_f32_lazy_weight.nx nx_f32_llama_block.nx nx_chat_warmcache.nx

diagram shows first 10 each side; +8 more imports, +0 more importers in the complete lists below.

imports: nx_syscalls.nxnx_tier.nxnx_bpe.nxnx_gguf.nxnx_gguf_load.nxnx_gguf_meta.nxnx_f32.nxnx_f32_kv_cache.nxnx_f32_lazy_weight.nxnx_f32_llama_block.nxnx_f32_llama_block_v4.nxnx_f32_llama_stack_v4.nxnx_f32_llama_layer_lazy_load.nxnx_f32_llm.nxnx_f32_llm_v4.nxnx_f32_llm_read_dims.nxnx_f32_bpe_load.nxnx_f32_llm_special_tokens.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap sys_read_file sys_openat_rd sys_lseek sys_mmap ↻ sys_read sys_close cw sys_write nx_gguf_parse nx_le_read_u32 nx_le_read_u64 nx_le_read_u32 ↻ _gguf_skip_value nx_le_read_u64 ↻ nx_le_read_u32 ↻ _gguf_skip_value ↻ sys_mmap ↻ nx_f32_llama_model_alloc sys_mmap ↻ nx_f32_llm_read_dims_from_ sys_mmap ↻ nx_gguf_meta_find nx_le_read_u64 ↻ nx_le_read_u32 ↻ _gmeta_bytes_equal nx_gguf_skip_value _gguf_skip_value ↻ nx_gguf_meta_read_string_l nx_le_read_u64 ↻ nx_gguf_meta_read_string_p _fld_read_arch_u32 sys_mmap ↻ _fld_concat_key nx_gguf_meta_find ↻ nx_gguf_meta_read_u32 nx_le_read_u32 ↻ _fld_read_vocab_size_from_ sys_mmap ↻

structs

40struct ChatCtx

consts

25const IM_MAGIC_67108864: i64 = 67108864
26const IM_MAGIC_262144: i64 = 262144
27const IM_MAGIC_524288: i64 = 524288
28const IM_MAGIC_2048: i64 = 2048
30const IM_START: i64 = 151644
31const IM_END: i64 = 151645
32const NL_TOK: i64 = 198
33const EPS: i64 = 0x358637BD
34const ASCALE: i64 = 0x3E000000
35const RBASE: i64 = 0x415D0EAB
36const SYS: *u8 = "system\nYou are Elara, a warm playful girlfriend. Keep replies short." as *u8
37const U1: *u8 = "user\nHi, I'm home." as *u8
38const U2: *u8 = "user\nWhat should we do tonight?" as *u8

functions

49func cw(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(1,s,n); return 0 }
called by 1: main calls 1: sys_write
50func cn(v: i64) -> i64 { let bb: *u8=sys_mmap(28); var m: i64=v; if m<0{m=0} let t: *u8=sys_mmap(28); var k: i64=0; if m==0{t[0]=48 as u8;k=1} while m>0{t[k]=(48+(m%10)) as u8;m=m/10;k=k+1} var i: i64=0; while i<k{bb[i]=t[k-1-i];i=i+1} sys_write(1,bb,k); return 0 }
called by 1: main calls 2: sys_mmapsys_write
51func slen(s: *u8) -> nx_int { var n: nx_int=0; while s[n]!=(0 as u8){n=n+1} return n }
called by 1: tok_seg
53func prefill_toks(cx: *ChatCtx, toks: *i64, n: nx_int) -> i64
called by 1: main calls 1: nx_f32_llm_forward_v4
63func tok_seg(cx: *ChatCtx, text: *u8, out: *i64) -> nx_int { return nx_bpe_encode_bytelevel(cx.vocab, text, slen(text), out) }
called by 1: main calls 2: nx_bpe_encode_bytelevelslen
65func amax(cx: *ChatCtx) -> nx_int
called by 1: generate
80func generate(cx: *ChatCtx, maxgen: nx_int) -> nx_int
called by 1: main calls 5: sys_mmapamaxnx_bpe_decode_bytelevelsys_writenx_f32_llm_forward_v4
99func main() -> i64
calls 16: sys_mmapsys_read_filecwnx_gguf_parsenx_f32_llama_model_allocnx_f32_llm_read_dims_from_gguf+10