code wiki / (root) / nx_zimage_flashattn_verify.nx

nx_zimage_flashattn_verify.nx

buildroot/runtime/nx_zimage_flashattn_verify.nx

6751 B155 linesdepth 5pulls 11 transitivereach 0 importersview sourcekind tooltopic zimage
docsdependenciesstructsconstsfunctions

about

nx_zimage_flashattn_verify.nx -- SOVEREIGN FlashAttention (online-softmax) SDPA, verified EXACT vs oracle. The O(n)-memory attention: instead of building the n x n score matrix, softmaxing, then weighting V, we STREAM keys into a running accumulator, rescaling it by exp(m_old - m_new) as the running max grows. Peak memory is O(d) per query (the accumulator + 3 scalars m,l,a), NOT O(n) -- so it scales to any n without materializing scores. This organ proves the recurrence is NUMERICALLY IDENTICAL to standard softmax attention (verifies vs the same oracle ao_pre as the direct SDPA). Lossless, not an approximation. Uses a LOCAL accumulator (no mmap scratch in the loop) per the nx_cc codegen lesson. No 3rd party. license_tier: ORIGINAL

dependencies 7 imports · 0 importers

nx_syscalls.nx nx_le.nx nx_f32.nx nx_f32_div.nx nx_f32_cvt.nx nx_f32_exp.nx nx_strconv.nx nx_zimage_flashattn_verify.nx

imports: nx_syscalls.nxnx_le.nxnx_f32.nxnx_f32_div.nxnx_f32_cvt.nxnx_f32_exp.nxnx_strconv.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main zfa_load sys_mmap sys_openat_rd sys_read sys_close nx_f32_div nx_f32_classify nx_f32_exp_field nx_f32_mant_field nx_f32_sign nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_i32_to_f32 nx_f32_sqrt nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_exp_field ↻ nx_f32_mant_field ↻ _f32_isqrt nx_le_read_u32 nx_f32_add nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_f32_mul nx_f32_classify ↻ nx_f32_sign ↻ nx_f32_mant_field ↻ nx_f32_exp_field ↻ nx_f32_lt nx_f32_is_nan nx_f32_classify ↻ nx_f32_is_zero nx_f32_classify ↻ nx_f32_exp nx_f32_classify ↻ nx_f32_sign ↻ _f32_to_i32_rne

structs

none

consts

17const K_MAGIC_3840: i64 = 3840
18const K_MAGIC_11520: i64 = 11520

functions

20func zfa_load(name: *u8, nl: i64, n_floats: i64) -> *u8
44func main() -> i64