code wiki / (root) / sketch_ddsketch.nx

sketch_ddsketch.nx

buildroot/runtime/sketch_ddsketch.nx

8931 B264 linesdepth 4pulls 4 transitivereach 2 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_ddsketch.nx -- DDSketch (Masson et al, PVLDB 2019). Multiplicative-error quantile sketch. Different error model than KLL (additive rank) and T-Digest (scale-function tail-tight): For query rank q, returned value v satisfies: true_value(q) ∈ [v · (1-α), v · (1+α)] I.e., the returned VALUE is within α-relative-error of the true. Perfect for log-distributed metrics (latency, sizes, durations). CAPABILITY: heavy-tail observability. Latency p99=100ms vs p99=1000ms both reported with ±1% multiplicative error, regardless of magnitude. ALGORITHM: bucket(x) = ⌈log_γ(x)⌉ where γ = (1+α)/(1-α) For our integer implementation: octave = bitlen(x) - 1 (floor log2) mantissa = x - (1 << octave) sub_bucket = mantissa * BPO / (1 << octave) bucket_index = octave * BPO + sub_bucket BPO = buckets-per-octave; relative error ≈ 1/BPO. BPO=64 -> ~1.5% multiplicative error BPO=128 -> ~0.8% MEMORY: 64 * BPO * 8 bytes (~32KB for BPO=64, 64KB for BPO=128). MERGE: count-wise addition; matching BPO required. LOSSLESS-LANGUAGE DISCIPLINE: nx_dd_query_quantile returns NX_ENV_REL_STDDEV with param_a = 1/BPO in PPB. Conf = 1e9 (deterministic). MaturityClass = ReferenceImpl.

dependencies 2 imports · 2 importers

syscalls.nx sketch_types.nx sketch_ddsketch.nx sketch_ddsketch_test.nx sketch_ddsketch_vs_tdigest_tail_be

imports: syscalls.nxsketch_types.nx

imported by: sketch_ddsketch_test.nxsketch_ddsketch_vs_tdigest_tail_bench.nx

structs

41struct DDSketch {

consts

37const NX_DD_BPO_MIN: i64 = 16
38const NX_DD_BPO_MAX: i64 = 1024
39const NX_DD_MAX_OCTAVE: i64 = 62 // covers x up to 2^62

functions

61func nx_dd_alloc(bpo: i64) -> *DDSketch {
called by 3: nx_dd_mergemainmain
98func nx_dd_bitlen(x: i64) -> i64 {
called by 1: nx_dd_bucket
117func nx_dd_bucket(d: *DDSketch, x: i64) -> i64 {
called by 1: nx_dd_add calls 1: nx_dd_bitlen
147func nx_dd_value_at_bucket(d: *DDSketch, b: i64) -> i64 {
called by 1: nx_dd_quantile
156func nx_dd_add(d: *DDSketch, x: i64) -> i64 {
called by 2: mainmain calls 1: nx_dd_bucket
185func nx_dd_quantile(d: *DDSketch, p_milli: i64) -> i64 {
216func nx_dd_query_quantile(d: *DDSketch, p_milli: i64) -> *ApproxI64 {
called by 1: main calls 2: nx_dd_quantilenx_approx_new
227func nx_dd_merge(a: *DDSketch, b: *DDSketch) -> *DDSketch {
called by 1: main calls 1: nx_dd_alloc
258func nx_dd_memory_bytes(d: *DDSketch) -> i64 {
262func nx_dd_total(d: *DDSketch) -> i64 {
called by 2: mainmain