code wiki / (root) / nx_flashattn_scale.nx

nx_flashattn_scale.nx

buildroot/runtime/nx_flashattn_scale.nx

4977 B126 linesdepth 5pulls 11 transitivereach 0 importersview sourcekind tool
docsdependenciesstructsconstsfunctions

about

nx_flashattn_scale.nx -- SOVEREIGN general-n FlashAttention, verified vs direct softmax on synthetic data. Proves the online-softmax recurrence scales to ARBITRARY n with O(d) memory: for each query we stream ALL n keys through a running (max m, normalizer l, accumulator acc[d]), rescaling acc by exp(m_old-m_new). The n x n score matrix is never built. Verifies bit-for-bit (within f32 tol) against a direct-softmax reference on the same synthetic Q/K/V. n=64,d=32 here; identical code runs at n=4096 with the SAME O(d) mem. license_tier: ORIGINAL

dependencies 7 imports · 0 importers

nx_syscalls.nx nx_le.nx nx_f32.nx nx_f32_div.nx nx_f32_cvt.nx nx_f32_exp.nx nx_strconv.nx nx_flashattn_scale.nx

imports: nx_syscalls.nxnx_le.nxnx_f32.nxnx_f32_div.nxnx_f32_cvt.nxnx_f32_exp.nxnx_strconv.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap fa_seed nx_f32_div nx_f32_classify nx_f32_exp_field nx_f32_mant_field nx_f32_sign nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_i32_to_f32 nx_f32_div ↻ nx_i32_to_f32 ↻ nx_f32_sqrt nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_exp_field ↻ nx_f32_mant_field ↻ _f32_isqrt nx_f32_add nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_f32_mul nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_f32_lt nx_f32_is_nan nx_f32_classify ↻ nx_f32_is_zero nx_f32_classify ↻ nx_f32_exp nx_f32_classify ↻ nx_f32_sign ↻ _f32_to_i32_rne nx_f32_classify ↻ nx_f32_sign ↻

structs

none

consts

none

functions

16func fa_seed(i: i64, dd: i64, a: i64, b: i64, m: i64) -> i64
called by 1: main calls 2: nx_f32_divnx_i32_to_f32
22func main() -> i64