sketch_ddsketch.nx
buildroot/runtime/sketch_ddsketch.nx
about
sketch_ddsketch.nx -- DDSketch (Masson et al, PVLDB 2019).
Multiplicative-error quantile sketch. Different error model than
KLL (additive rank) and T-Digest (scale-function tail-tight):
For query rank q, returned value v satisfies:
true_value(q) ∈ [v · (1-α), v · (1+α)]
I.e., the returned VALUE is within α-relative-error of the true.
Perfect for log-distributed metrics (latency, sizes, durations).
CAPABILITY: heavy-tail observability. Latency p99=100ms vs p99=1000ms
both reported with ±1% multiplicative error, regardless of magnitude.
ALGORITHM:
bucket(x) = ⌈log_γ(x)⌉ where γ = (1+α)/(1-α)
For our integer implementation:
octave = bitlen(x) - 1 (floor log2)
mantissa = x - (1 << octave)
sub_bucket = mantissa * BPO / (1 << octave)
bucket_index = octave * BPO + sub_bucket
BPO = buckets-per-octave; relative error ≈ 1/BPO.
BPO=64 -> ~1.5% multiplicative error
BPO=128 -> ~0.8%
MEMORY: 64 * BPO * 8 bytes (~32KB for BPO=64, 64KB for BPO=128).
MERGE: count-wise addition; matching BPO required.
LOSSLESS-LANGUAGE DISCIPLINE:
nx_dd_query_quantile returns NX_ENV_REL_STDDEV with param_a = 1/BPO
in PPB. Conf = 1e9 (deterministic). MaturityClass = ReferenceImpl.
dependencies 2 imports · 2 importers
imports: syscalls.nxsketch_types.nx
imported by: sketch_ddsketch_test.nxsketch_ddsketch_vs_tdigest_tail_bench.nx
structs
| 41 | struct DDSketch { |
consts
| 37 | const NX_DD_BPO_MIN: i64 = 16 |
| 38 | const NX_DD_BPO_MAX: i64 = 1024 |
| 39 | const NX_DD_MAX_OCTAVE: i64 = 62 // covers x up to 2^62 |
functions
| 61 | func nx_dd_alloc(bpo: i64) -> *DDSketch { |
| 98 | func nx_dd_bitlen(x: i64) -> i64 {
called by 1: nx_dd_bucket |
| 117 | func nx_dd_bucket(d: *DDSketch, x: i64) -> i64 { |
| 147 | func nx_dd_value_at_bucket(d: *DDSketch, b: i64) -> i64 {
called by 1: nx_dd_quantile |
| 156 | func nx_dd_add(d: *DDSketch, x: i64) -> i64 { |
| 185 | func nx_dd_quantile(d: *DDSketch, p_milli: i64) -> i64 { |
| 216 | func nx_dd_query_quantile(d: *DDSketch, p_milli: i64) -> *ApproxI64 { |
| 227 | func nx_dd_merge(a: *DDSketch, b: *DDSketch) -> *DDSketch { |
| 258 | func nx_dd_memory_bytes(d: *DDSketch) -> i64 { |
| 262 | func nx_dd_total(d: *DDSketch) -> i64 { |