nx_f32_llm_run_v2.nx
buildroot/runtime/nx_f32_llm_run_v2.nx
about
nx_f32_llm_run_v2.nx -- production runner with full sampler config.
Wraps the v1 runner pattern but accepts a sampler config struct
covering temperature + top-k + top-p + repetition penalty, and
maintains a recent-tokens ring buffer that feeds the penalty
across decode steps.
Composes:
nx_bpe_encode / nx_bpe_decode
nx_f32_llm_forward
nx_f32_sampler_apply_repetition_penalty
nx_f32_sampler_sample_top_p / top_k / temp / argmax
nx_f32_kv_cache_alloc/append/advance
genealogy_id: standard_decode_loop_with_sampler_pipeline
lineage_id: substrate_f32_llm_run_v2
dependencies 7 imports · 12 importers
diagram shows first 10 each side; +0 more imports, +2 more importers in the complete lists below.
imports: nx_syscalls.nxnx_tier.nxnx_bpe.nxnx_f32.nxnx_f32_kv_cache.nxnx_f32_llm.nxnx_f32_sampler.nx
imported by: nx_coder_gen_f32.nxnx_f32_llm_bench_test.nxnx_f32_llm_cohere_test.nxnx_f32_llm_live_gen_test.nxnx_f32_llm_run_v2_test.nxnx_f32_llm_run_v3.nxnx_f32_llm_run_v3_test.nxnx_f32_llm_serve.nxnx_f32_qwen_capital_gate.nxnx_llm_france_probe.nxnx_qwen_native_gate.nxnx_qwen_wsl_timing_gate.nx
structs
| 32 | struct NxF32SamplerCfg |
consts
| 25 | const NX_MAGIC_16384: i64 = 16384 |
| 27 | const NX_FRN2_OK_BASE: nx_int = 0 |
| 28 | const NX_FRN2_ERR_NULL: nx_int = -1 |
| 29 | const NX_FRN2_ERR_BAD_DIM: nx_int = -2 |
| 30 | const NX_FRN2_ERR_FORWARD: nx_int = -3 |
| 40 | const NX_F32_SAMPLER_CFG_BYTES: nx_int = 40 // 5 fields * 8 |
functions
| 42 | func nx_f32_sampler_cfg_alloc() -> *NxF32SamplerCfg |
| 54 | func _frn2_pick(cfg: *NxF32SamplerCfg, logits: *i64, vocab_size: nx_int, called by 1: nx_f32_llm_run_v2 calls 3: nx_f32_sampler_argmaxnx_f32_sampler_sample_top_pnx_f32_sampler_sample_top_k |
| 72 | func _frn2_apply_penalty(cfg: *NxF32SamplerCfg, logits: *i64, vocab_size: nx_int, |
| 84 | func _frn2_recent_push(recent: *i64, recent_cap: nx_int, called by 1: nx_f32_llm_run_v2 |
| 100 | func nx_f32_llm_run_v2(model: *NxF32LlamaModel, called by 1: main calls 7: sys_mmapnx_bpe_encode_frn2_recent_pushnx_f32_llm_forward_frn2_apply_penalty_frn2_pick+1 |