code wiki / (root) / nx_f32_llm_run.nx

nx_f32_llm_run.nx

buildroot/runtime/nx_f32_llm_run.nx

6090 B155 linesdepth 11pulls 28 transitivereach 1 importersview sourcekind librarytopic f32
docsdependenciesstructsconstsfunctions

about

nx_f32_llm_run.nx -- autoregressive generation loop runner. The capstone API surface: takes a populated model + vocab + cache, a prompt as raw bytes, and produces generated bytes via prefill + repeated (sample, decode-forward) loop. Composes every prior brick in the f32 LLM pipeline: nx_bpe_encode prompt bytes -> token ids nx_f32_llm_forward prefill + per-token decode forward nx_f32_sampler_sample_top_k (or argmax if top_k <= 0) nx_bpe_decode token id -> output bytes nx_f32_kv_cache growing cache across forward calls API: nx_f32_llm_run(model, vocab, cache, prompt, n_prompt, max_new_tokens, inv_temp_f32, top_k, eps, attn_scale, rope_log_base, apply_rope, prng_state, eos_token_id, out_bytes, out_bytes_cap) -> nx_int (= n bytes emitted) 16 args -- at the NishiLang argument limit. Termination: * eos_token_id matched -> stop (caller passes -1 to disable) * max_new_tokens reached -> stop * out_bytes_cap reached -> stop Returns the count of bytes emitted into out_bytes (>= 0), or a negative error verdict. genealogy_id: standard_autoregressive_decode_loop lineage_id: substrate_f32_llm_run_v1

dependencies 6 imports · 1 importers

nx_syscalls.nx nx_tier.nx nx_bpe.nx nx_f32_kv_cache.nx nx_f32_llm.nx nx_f32_sampler.nx nx_f32_llm_run.nx nx_f32_llm_run_test.nx

imports: nx_syscalls.nxnx_tier.nxnx_bpe.nxnx_f32_kv_cache.nxnx_f32_llm.nxnx_f32_sampler.nx

imported by: nx_f32_llm_run_test.nx

structs

none

consts

41const NX_FRN_OK_BASE: nx_int = 0 // success: emitted bytes count returned
42const NX_FRN_ERR_NULL: nx_int = -1
43const NX_FRN_ERR_BAD_DIM: nx_int = -2
44const NX_FRN_ERR_FORWARD: nx_int = -3
45const NX_FRN_ERR_TOK_OVF: nx_int = -4

functions

49func nx_f32_llm_run_pick(logits: *i64, vocab_size: nx_int,
62func nx_f32_llm_run(model: *NxF32LlamaModel,