nx_token_sample.nx

nx_token_sample.nx -- logit sampling primitives for inference. Closes the OUTPUT half of the transformer inference loop. Composes against shipped primitives -- no new math: nx_prng (canonical RNG) nx_attention._attn_exp_q10 (numerically-stable softmax) Standard top-K mask + categorical sample math The typical decoder-only LLM inference loop: for each generation step: logits = forward_pass(...)[-1, :] // last token, vocab-wide nx_logit_apply_temperature(logits, vocab, temp_q10) nx_logit_top_k_mask(logits, vocab, K) softmax(logits) // (caller via nx_attn_softmax_row_q10 // reshaping to [1, vocab]) next_token = nx_sample_categorical(probs, vocab, prng_state) Top-P (nucleus) sampling queued -- needs a sort primitive (or partial-sort) which isn't shipped yet. Per the no-skipping cardinal, top-P lands after nx_sort. ===== Math ======================================================= Temperature: logit_i' = logit_i * Q10 / temp_q10 * temp = Q10 (1.0) -> unchanged * temp = Q10/2 (0.5) -> sharper distribution (sampled = more deterministic) * temp = 2*Q10 (2.0) -> flatter distribution (more random) Top-K masking: keep K highest logits, set rest to NEG_INF so they get 0 weight in subsequent softmax. Implemented via repeated max-find -- O(n*k); for K=40 on 50k vocab that's 2M ops, fine. Categorical sample: given probs in Q10 that nominally sum to Q10, draw u in [0, sum_q10) and walk CDF. Returns index where the CDF crosses u. Per the bounded-loop + bits-up cardinals.