code wiki / (root) / nx_f32_llm_serve.nx

nx_f32_llm_serve.nx

buildroot/runtime/nx_f32_llm_serve.nx

17315 B351 linesdepth 14pulls 68 transitivereach 0 importersview sourcekind servicetopic f32
docsdependenciesstructsconstsfunctions

about

nx_f32_llm_serve.nx -- the SOVEREIGN LLM generation SEAT. Loads real Qwen2.5-0.5B GGUF ONCE at startup, binds 127.0.0.1:11434 (the seam nx_writehub_seat already POSTs to), and on POST /chat runs nx_f32_llm_run_v3 -> returns OpenAI-chat-shaped JSON. Drop-in replacement for the Docker gen-llm:11434 seat -- 100% sovereign (our nx_f32 stack + our own HTTP accept-loop, the nx_writer_serve pattern), NO Docker, NO 3rd-party lib, no shell. NOT yet supervised on this box (no nishi_supervisor.sh here -- launch via knowledge/shell/seat_probe.sh or setsid; supervision = follow-on when the seat moves to its target host). 127.0.0.1-only = seam by construction. PERF CAVEAT (honest): scalar-CPU matmul + re-dequant per token -> slow; MAX_NEW capped low. The engine's COHERENCE is gated separately by nx_f32_llm_live_gen_test; this organ is the serving layer around it. expect: LIVE on :11434 (blocks). license_tier: ORIGINAL module: nishi-core.seat.llm

dependencies 27 imports · 0 importers

nx_syscalls.nx nx_connect.nx nx_tier.nx nx_bpe.nx nx_gguf.nx nx_gguf_load.nx nx_gguf_meta.nx nx_f32.nx nx_f32_kv_cache.nx nx_f32_lazy_weight.nx nx_f32_llm_serve.nx

diagram shows first 10 each side; +17 more imports, +0 more importers in the complete lists below.

imports: nx_syscalls.nxnx_connect.nxnx_tier.nxnx_bpe.nxnx_gguf.nxnx_gguf_load.nxnx_gguf_meta.nxnx_f32.nxnx_f32_kv_cache.nxnx_f32_lazy_weight.nxnx_f32_llama_block.nxnx_f32_llama_block_v4.nxnx_f32_llama_stack_v4.nxnx_f32_llama_layer_lazy_load.nxnx_f32_llm.nxnx_f32_llm_v4.nxnx_f32_llm_read_dims.nxnx_f32_bpe_load.nxnx_f32_llm_special_tokens.nxnx_f32_sampler.nxnx_f32_llm_run_v2.nxnx_f32_llm_run_v3.nxnx_http_client.nxnx_f32_div.nxnx_f32_cvt.nxnx_prng.nxnx_reasoning.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main ls_puts sys_write sys_mmap sys_read_file sys_openat_rd sys_lseek sys_mmap ↻ sys_read sys_close sys_exit nx_gguf_parse nx_le_read_u32 nx_le_read_u64 nx_le_read_u32 ↻ _gguf_skip_value nx_le_read_u64 ↻ nx_le_read_u32 ↻ _gguf_skip_value ↻ sys_mmap ↻ nx_f32_llama_model_alloc sys_mmap ↻ nx_f32_llm_read_dims_from_ sys_mmap ↻ nx_gguf_meta_find nx_le_read_u64 ↻ nx_le_read_u32 ↻ _gmeta_bytes_equal nx_gguf_skip_value _gguf_skip_value ↻ nx_gguf_meta_read_string_l nx_le_read_u64 ↻ nx_gguf_meta_read_string_p _fld_read_arch_u32 sys_mmap ↻ _fld_concat_key nx_gguf_meta_find ↻ nx_gguf_meta_read_u32 nx_le_read_u32 ↻ _fld_read_vocab_size_from_

structs

none

consts

37const NX_MAGIC_67108864: i64 = 67108864
38const NX_MAGIC_262144: i64 = 262144
39const NX_MAGIC_524288: i64 = 524288
40const NX_MAGIC_151644: i64 = 151644
41const NX_MAGIC_151645: i64 = 151645
42const NX_MAGIC_1048576: i64 = 1048576
43const NX_MAGIC_65536: i64 = 65536
44const NX_MAGIC_131072: i64 = 131072
45const NX_MAGIC_200000: i64 = 200000
46const NX_MAGIC_65534: i64 = 65534
50const NX_LSEAT_PORT: i64 = 11434

functions

52func ls_puts(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} sys_write(1,s,n); return 0 }
called by 1: main calls 1: sys_write
53func ls_slen(s: *u8) -> i64 { var n: i64=0; while s[n]!=(0 as u8){n=n+1} return n }
called by 4: ls_body_intls_body_temp_tenthsls_extract_promptmain
54func ls_find(buf: *u8, n: i64, ndl: *u8, nl: i64) -> i64
called by 5: ls_body_intls_body_temp_tenthsls_read_requestls_extract_promptmain
60func ls_cat(dst: *u8, off: i64, s: *u8) -> i64 { var i: i64=0; while s[i]!=(0 as u8){ dst[off]=s[i]; off=off+1; i=i+1 } return off }
called by 1: main
61func ls_write_all(fd: i64, buf: *u8, n: i64) -> i64 { var off: i64=0; while off<n { let w: i64=sys_write(fd, ((buf as i64)+off) as *u8, n-off); if w<=0 { off=n } else { off=off+w } } return 0 }
called by 1: main calls 1: sys_write
62func ls_atoi(s: *u8) -> i64
called by 1: ls_read_request
76func ls_body_int(body: *u8, blen: i64, key: *u8) -> i64
called by 1: main calls 2: ls_slenls_find
97func ls_body_temp_tenths(body: *u8, blen: i64) -> i64
called by 1: main calls 2: ls_slenls_find
134func ls_read_request(fd: i64, req: *u8, cap: i64) -> i64
called by 1: main calls 3: sys_readls_findls_atoi
153func ls_extract_prompt(body: *u8, blen: i64, out: *u8, cap: i64) -> i64
called by 1: main calls 2: ls_slenls_find
175func ls_json_esc(src: *u8, sn: i64, dst: *u8, cap: i64) -> i64
called by 1: main
190func main(argc: i64, argv: *i64) -> i64
calls 37: ls_putssys_mmapsys_read_filesys_exitnx_gguf_parsenx_f32_llama_model_alloc+31