code wiki / (root) / sketch_reqsketch.nx

sketch_reqsketch.nx

buildroot/runtime/sketch_reqsketch.nx

12726 B380 linesdepth 4pulls 4 transitivereach 2 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_reqsketch.nx -- relative-error streaming quantile sketch. Cormode-Karnin-Liberty-Thaler-Vesely 2021, "Relative Error Streaming Quantiles" (FOCS / VLDB). ReqSketch tightens KLL on tail queries: where KLL bounds |rank_est - rank_true| <= eps * N (additive in N), ReqSketch bounds |rank_est - rank_true| <= eps * min(rank_true, N - rank_true) (multiplicative in the rank's distance from the nearer extreme). Tail quantiles (p99 / p99.9 / p0.1) get strictly tighter guarantees as N grows. COMPLEMENTS KLL + T-DIGEST: - KLL: provable uniform additive bound; loose at tails. - T-Digest: empirically tail-tight; no global rank guarantee. - ReqSketch: provable multiplicative tail bound; strictly stronger than KLL for tail queries at large N. CONSTRUCTION (simplified reference impl, hra = "high-rank- accurate" upper-tail-tight variant -- mirror with hra=0 for lower-tail-tight): Compactor cascade like KLL, but the compact() rule preserves the upper section of each compactor exactly (no down-sampling) while KLL-coin-flipping the lower section. Section size shrinks linearly with level -- at level h, preserve_count_h = max(1, k - 2*h), so at high levels almost all items get evicted, while at low levels half stay in place. Items preserved at level h carry weight 2^h same as KLL; coin-flipped survivors get weight 2^(h+1) upon promotion. SIMPLIFICATION VS PAPER (honest scope): Paper uses numSections in {3, 6, 12, ...} growing geometrically with N and section_size = sqrt(2) * k_base * 2^(-h/2). We use numSections=2 fixed and section_size = max(1, k/2 - h) linear shrinkage. Same asymptotic shape (eps ~ 1/sqrt(k)), slightly looser constant. v2 will swap in the geometric schedule. LOSSLESS-LANGUAGE DISCIPLINE (doc 20): Query returns ApproxI64 with envelope_kind = NX_ENV_REL_RANK_ERROR, param_a = eps_ppb (relative rank error * 1e9), conf = 0.95, maturity = ReferenceImpl, adv = Honest.

dependencies 2 imports · 2 importers

syscalls.nx sketch_types.nx sketch_reqsketch.nx sketch_reqsketch_test.nx sketch_reqsketch_vs_kll_bench.nx

imports: syscalls.nxsketch_types.nx

imported by: sketch_reqsketch_test.nxsketch_reqsketch_vs_kll_bench.nx

structs

57struct Req

consts

45const NX_REQ_K_MIN: i64 = 8
46const NX_REQ_K_MAX: i64 = 1024
47const NX_REQ_MAX_LEVELS: i64 = 24
50const NX_REQ_LCG_A: i64 = 1103515245
51const NX_REQ_LCG_C: i64 = 12345
52const NX_REQ_LCG_MOD: i64 = 0x7FFFFFFF

functions

73func nx_req_alloc(k: i64, hra: i64, seed: i64) -> *Req
called by 2: mainmain
103func nx_req_rng_next(s: *Req) -> i64
called by 1: nx_req_compact
110func nx_req_level_addr(s: *Req, level: i64) -> *i64
118func nx_req_preserve_count(s: *Req, level: i64) -> i64
called by 1: nx_req_compact
126func nx_req_sort_level(s: *Req, level: i64) -> i64
called by 1: nx_req_compact calls 1: nx_req_level_addr
162func nx_req_compact(s: *Req, level: i64) -> i64
230func nx_req_add(s: *Req, value: i64) -> i64
247func nx_req_total_weight(s: *Req) -> i64
262func nx_req_quantile(s: *Req, p_milli: i64) -> i64
319func nx_req_rank(s: *Req, value: i64) -> i64
355func nx_req_rel_rank_error_ppb(k: i64) -> i64
363func nx_req_query_quantile(s: *Req, p_milli: i64) -> *ApproxI64
374func nx_req_memory_bytes(s: *Req) -> i64
called by 1: main
378func nx_req_levels_used(s: *Req) -> i64
called by 1: main