code wiki / (root) / sketch_kll.nx

sketch_kll.nx

buildroot/runtime/sketch_kll.nx

11269 B325 linesdepth 4pulls 4 transitivereach 3 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_kll.nx -- compactor-hierarchy quantile sketch. MRL99-style (Manku-Rajagopalan-Lindsay 1999) "compactor cascade": maintain levels 0, 1, 2, ... each holding up to k items. Items at level h carry weight 2^h. When level h fills, sort it, flip a coin, promote either even-indexed or odd-indexed items to level h+1 (the other half discarded). Karnin-Lang-Liberty 2016 (FOCS) tightens this via geometric per-level capacity decay; we use uniform capacity here for simpler code at slightly looser bound. Rank-error declared conservatively as ~1/sqrt(k) at conf 0.95. Versus Reservoir (sketch_reservoir.nx) HONEST COMPARISON (2026-05-20 audit, sketch_kll_vs_reservoir_bench.nx): - At MATCHED slot count, Reservoir wins on RANK-ERROR magnitude. KLL k=64 declared eps=0.12 vs Reservoir cap=1024 declared eps=0.0425 (both ~8KB). Empirically Reservoir wins on uniform 1..10000 stream too. - KLL's structural advantages live ELSEWHERE: (1) DETERMINISTIC mergeable across instances (Reservoir merge requires algorithmic choices that change semantics) (2) O(1) amortized update at any N (Reservoir's update is O(1/N) probability; quickly stops sampling at large N) (3) Cascade preserves rank info from FULL stream history (Reservoir's sample is a fixed random subset) - Earlier doc claim of "tighter rank-error at fixed memory" was a misframe (compared at same k, not same memory). Corrected. LOSSLESS-LANGUAGE DISCIPLINE (doc 20): Same NX_ENV_RANK_ERROR envelope as Reservoir; substrate doesn't double-count. Multiple primitives sharing an envelope kind is fine -- the discriminator is about the SHAPE of the error, not uniqueness.

dependencies 2 imports · 3 importers

syscalls.nx sketch_types.nx sketch_kll.nx sketch_kll_test.nx sketch_kll_vs_reservoir_bench.nx sketch_reqsketch_vs_kll_bench.nx

imports: syscalls.nxsketch_types.nx

imported by: sketch_kll_test.nxsketch_kll_vs_reservoir_bench.nxsketch_reqsketch_vs_kll_bench.nx

structs

48struct Kll {

consts

39const NX_KLL_K_MIN: i64 = 8
40const NX_KLL_K_MAX: i64 = 10000
41const NX_KLL_MAX_LEVELS: i64 = 16
44const NX_KLL_LCG_A: i64 = 1103515245
45const NX_KLL_LCG_C: i64 = 12345
46const NX_KLL_LCG_MOD: i64 = 0x7FFFFFFF

functions

65func nx_kll_alloc(k: i64, seed: i64) -> *Kll {
called by 3: mainmainmain
94func nx_kll_rng_next(s: *Kll) -> i64 {
called by 1: nx_kll_compact
103func nx_kll_level_addr(s: *Kll, level: i64) -> *i64 {
107func nx_kll_sort_level(s: *Kll, level: i64) -> i64 {
called by 1: nx_kll_compact calls 1: nx_kll_level_addr
134func nx_kll_compact(s: *Kll, level: i64) -> i64 {
175func nx_kll_add(s: *Kll, value: i64) -> i64 {
191func nx_kll_total_weight(s: *Kll) -> i64 {
208func nx_kll_quantile(s: *Kll, p_milli: i64) -> i64 {
268func nx_kll_rank(s: *Kll, value: i64) -> i64 {
299func nx_kll_rank_error_ppb(k: i64) -> i64 {
308func nx_kll_query_quantile(s: *Kll, p_milli: i64) -> *ApproxI64 {
319func nx_kll_memory_bytes(s: *Kll) -> i64 {
called by 2: mainmain
323func nx_kll_levels_used(s: *Kll) -> i64 {
called by 1: main