nx_f32_llm_run.nx
buildroot/runtime/nx_f32_llm_run.nx
about
nx_f32_llm_run.nx -- autoregressive generation loop runner.
The capstone API surface: takes a populated model + vocab + cache,
a prompt as raw bytes, and produces generated bytes via prefill +
repeated (sample, decode-forward) loop.
Composes every prior brick in the f32 LLM pipeline:
nx_bpe_encode prompt bytes -> token ids
nx_f32_llm_forward prefill + per-token decode forward
nx_f32_sampler_sample_top_k (or argmax if top_k <= 0)
nx_bpe_decode token id -> output bytes
nx_f32_kv_cache growing cache across forward calls
API:
nx_f32_llm_run(model, vocab, cache, prompt, n_prompt,
max_new_tokens, inv_temp_f32, top_k,
eps, attn_scale, rope_log_base, apply_rope,
prng_state, eos_token_id, out_bytes,
out_bytes_cap) -> nx_int (= n bytes emitted)
16 args -- at the NishiLang argument limit.
Termination:
* eos_token_id matched -> stop (caller passes -1 to disable)
* max_new_tokens reached -> stop
* out_bytes_cap reached -> stop
Returns the count of bytes emitted into out_bytes (>= 0), or a
negative error verdict.
genealogy_id: standard_autoregressive_decode_loop
lineage_id: substrate_f32_llm_run_v1
dependencies 6 imports · 1 importers
imports: nx_syscalls.nxnx_tier.nxnx_bpe.nxnx_f32_kv_cache.nxnx_f32_llm.nxnx_f32_sampler.nx
imported by: nx_f32_llm_run_test.nx
structs
| none |
consts
| 41 | const NX_FRN_OK_BASE: nx_int = 0 // success: emitted bytes count returned |
| 42 | const NX_FRN_ERR_NULL: nx_int = -1 |
| 43 | const NX_FRN_ERR_BAD_DIM: nx_int = -2 |
| 44 | const NX_FRN_ERR_FORWARD: nx_int = -3 |
| 45 | const NX_FRN_ERR_TOK_OVF: nx_int = -4 |
functions
| 49 | func nx_f32_llm_run_pick(logits: *i64, vocab_size: nx_int, |
| 62 | func nx_f32_llm_run(model: *NxF32LlamaModel, |