code wiki / (root) / nx_speculative_verify.nx

nx_speculative_verify.nx

buildroot/runtime/nx_speculative_verify.nx

9538 B231 linesdepth 2pulls 2 transitivereach 2 importersview sourcekind library
docsdependenciesstructsconstsfunctions

about

nx_speculative_verify.nx -- H3 speculative decoding bits-up. Per NISHI_ELDER_AI_OFF_DOCKER_2026_05_20.md ยง2.1 H3: small fast draft model proposes K tokens; large target model verifies them in one parallel forward pass; substrate keeps the longest accept-prefix + 1 bonus token (the target's own next-token prediction at the rejection boundary is free). Leviathan 2023 + Chen 2023 "Accelerating Large Language Model Decoding with Speculative Sampling." Practical wins: 2-3x throughput at equivalent quality when draft + target agree on common tokens. V1 mechanics: the substrate primitive does NOT run a model. It accepts the draft tokens (what the small model proposed) and a parallel accept_flag array (1 == target verified, 0 == target rejected) computed upstream by the verification kernel, PLUS the target's own next-token at the rejection boundary (the "free" bonus token). It returns the longest accept-prefix length and the count of tokens actually committed to the caller's output buffer. Composition path: - nx_speculative_session_new(K, request_id) on admit - For each verification round: nx_speculative_verify(session, draft_tokens, accept_flags, K_actual, target_next_token, out_committed) returns count_committed - Per-session stats (drafted / accepted / rounds) accumulate so the consumer can monitor the acceptance rate and back off speculative depth when the draft model is mispredicting - Acceptance rate exposed as Q16 fixed-point (rate * 65536) to keep the substrate primitive integer-only Pure substrate logic. No Linux features. Composes with shipped nx_batch_scheduler (each batch slot can carry a NxSpeculativeSession pointer for its decoding state). V1 honest scope: - Fixed max-K (NX_SPEC_MAX_K = 16) per round - Caller provides verification flags (substrate doesn't run the target forward pass; that's a tensor kernel one layer up) - Greedy verification only (no sampled-rejection-resampling;

dependencies 1 imports · 2 importers

nx_syscalls.nx nx_speculative_verify.nx nx_hackers_algo_compose_test.nx nx_speculative_verify_test.nx

imports: nx_syscalls.nx

imported by: nx_hackers_algo_compose_test.nxnx_speculative_verify_test.nx

structs

97struct NxSpeculativeSession

consts

74const NX_MAGIC_65536: i64 = 65536
77const NX_SPEC_MAX_K: i64 = 16
80const NX_SPEC_OK: i64 = 0
81const NX_SPEC_BAD_INPUT: i64 = 1
82const NX_SPEC_BAD_K: i64 = 2
83const NX_SPEC_TAMPER: i64 = 3
84const NX_SPEC_N_VERDICTS: i64 = 4
93const NX_SPEC_SESSION_CANARY_PRE: i64 = 0x537065635365737A // "SpecSesz"
94const NX_SPEC_SESSION_CANARY_POST: i64 = 0x53657373456E6464 // "SessEndd"

functions

86func nx_spec_verdict_is_valid(v: i64) -> i64
called by 1: main
108func nx_spec_session_is_valid(s: *NxSpeculativeSession) -> i64
124func nx_speculative_session_new(request_id: i64, max_k: i64) -> *NxSpeculativeSession
called by 2: mainmain calls 1: sys_mmap
154func nx_speculative_verify(s: *NxSpeculativeSession, draft_tokens: *i64, accept_flags: *i64, k_actual: i64, target_next: i64, out_committed: *i64) -> i64
called by 2: mainmain calls 1: nx_spec_session_is_valid
194func nx_spec_total_drafted(s: *NxSpeculativeSession) -> i64
called by 2: mainmain calls 1: nx_spec_session_is_valid
199func nx_spec_total_accepted(s: *NxSpeculativeSession) -> i64
called by 2: mainmain calls 1: nx_spec_session_is_valid
204func nx_spec_total_committed(s: *NxSpeculativeSession) -> i64
called by 1: main calls 1: nx_spec_session_is_valid
209func nx_spec_total_rounds(s: *NxSpeculativeSession) -> i64
called by 1: main calls 1: nx_spec_session_is_valid
216func nx_spec_acceptance_rate_q16(s: *NxSpeculativeSession) -> i64
called by 1: main calls 1: nx_spec_session_is_valid
223func nx_spec_session_request_id(s: *NxSpeculativeSession) -> i64
called by 1: main calls 1: nx_spec_session_is_valid
228func nx_spec_session_max_k(s: *NxSpeculativeSession) -> i64
called by 1: main calls 1: nx_spec_session_is_valid