code wiki / (root) / nx_llm_run_v2.nx

nx_llm_run_v2.nx

buildroot/runtime/nx_llm_run_v2.nx

9214 B217 linesdepth 9pulls 33 transitivereach 18 importersview sourcekind librarytopic llm
docsdependenciesstructsconstsfunctions

about

nx_llm_run_v2.nx -- end-to-end Llama-class inference composer (real weights edition). L6 brick. The CLOSER for the inference path: composes every loader + per-layer compute + sampler primitive shipped this session into a single callable that maps: text prompt + GGUF + BPE vocab + sampling knobs -> next-token-id Compared to nx_llm_run.nx (v1, scaffold-only): * v1 omitted the per-layer transformer compute (no weight loader) * v1 simulated output projection as identity copy * v2 sources ALL weights from a parsed GGUF and runs the full forward stack Per the bits-up + no-skipping cardinals: no new math, no reinvented kernels. Pure composition of: nx_bpe.nx_bpe_encode -- text -> token ids nx_gguf_load_model.nx_..._weights -- top-level tensor bundle nx_tensor.nx_t_alloc -- working buffers nx_embedding.nx_embedding_lookup -- token-id -> hidden vector nx_transformer_stack.nx_..._fwd -- N-layer forward nx_blas_i64.nx_blas_matmul -- output projection to vocab nx_token_sample.* + nx_attention.nx_attn_softmax_row_q10 -- temp + top_k + softmax + sample genealogy_id: standard_decoder_only_llm_inference + touvron_2023_llama + radford_2019_gpt2 lineage_id: substrate_llm_run_v2_real_weights

dependencies 15 imports · 4 importers

nx_syscalls.nx nx_tier.nx nx_loop.nx nx_tensor.nx nx_bpe.nx nx_embedding.nx nx_blas_i64.nx nx_attention.nx nx_token_sample.nx nx_model_spec.nx nx_llm_run_v2.nx nx_actor_role_llm_v2.nx nx_actor_role_llm_v2_real_test.nx nx_gguf_fixture_tiny_test.nx nx_llm_run_v2_test.nx

diagram shows first 10 each side; +5 more imports, +0 more importers in the complete lists below.

imports: nx_syscalls.nxnx_tier.nxnx_loop.nxnx_tensor.nxnx_bpe.nxnx_embedding.nxnx_blas_i64.nxnx_attention.nxnx_token_sample.nxnx_model_spec.nxnx_prng.nxnx_gguf.nxnx_gguf_load.nxnx_gguf_load_model.nxnx_transformer_stack.nx

imported by: nx_actor_role_llm_v2.nxnx_actor_role_llm_v2_real_test.nxnx_gguf_fixture_tiny_test.nxnx_llm_run_v2_test.nx

structs

none

consts

69const NX_LR2_OK: nx_int = 0
70const NX_LR2_ERR_BAD_SPEC: nx_int = 1
71const NX_LR2_ERR_TOKENIZE: nx_int = 2
72const NX_LR2_ERR_MODEL_LOAD: nx_int = 3
73const NX_LR2_ERR_EMBED: nx_int = 4
74const NX_LR2_ERR_STACK_FWD: nx_int = 5
75const NX_LR2_ERR_TIED_NOT_SUPPORTED: nx_int = 6
76const NX_LR2_ERR_OOM: nx_int = 7
77const NX_LR2_ERR_BAD_ROPE: nx_int = 8
78const NX_LR2_N_VERDICTS: nx_int = 9

functions

80func nx_lr2_verdict_is_valid(v: nx_int) -> nx_int
called by 1: main
116func nx_llm_generate_one_v2(