code wiki / (root) / sketch_tdigest.nx

sketch_tdigest.nx

buildroot/runtime/sketch_tdigest.nx

12612 B340 linesdepth 4pulls 4 transitivereach 8 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_tdigest.nx -- T-Digest (Dunning 2019) tail-tight quantile sketch. Maintain centroids (mean, weight) sorted by mean. Each centroid covers a window of the cumulative-distribution domain [q_left, q_right]. The MAX WEIGHT a centroid may absorb depends on its q-position via a scale function that gives more room to centroids in the middle of the distribution and TIGHTLY BOUNDS centroids near the tails -- the headline T-Digest property. COMPLEMENTS KLL: - KLL: provable uniform rank-error bound, lighter at middle quantiles, looser at tails. - T-Digest: empirically tail-tight (p99/p99.9 accuracy is the headline), no global rank-error guarantee. Together: both shipping means callers pick the right tool per query. DUNNING k1 vs SIMPLIFIED SCALE FUNCTION (v1 trade-off): Dunning's k1(q) = (delta / 2π) · arcsin(2q - 1) → w_max_norm(q) = k1_inv(k1(q) + 1) - q exact: tail w_max ~ sqrt-like; middle ~ delta-bounded. v1 here: w_max_norm(q) = 4·q·(1-q) · π / delta The simplified scale UNDERESTIMATES tail capacity (more conservative = MORE centroids near tails, better tail accuracy than spec). At q=0.5 it gives 0.0314·N matching Dunning within 1%. Trade-off: memory uses slightly more centroids than necessary; queried accuracy is no worse than Dunning's bound, often better at tails. v2 will swap in a tabulated arcsin-based scale. LOSSLESS-LANGUAGE DISCIPLINE (doc 20): Query returns ApproxI64 with envelope_kind = NX_ENV_RANK_ERROR, param_a = ~1% absolute rank error (conservative bound for delta=100), maturity = ReferenceImpl, adv = Honest.

dependencies 2 imports · 8 importers

syscalls.nx sketch_types.nx sketch_tdigest.nx _a2a_td_x100.nx _a2a_tdigest_subst.nx _py_quant_substrate.nx sketch_ddsketch_vs_tdigest_tail_be sketch_observability_dashboard_ben sketch_tdigest_test.nx sketch_tdigest_v2_vs_v1_bench.nx sketch_tdigest_vs_materialized_qua

imports: syscalls.nxsketch_types.nx

imported by: _a2a_td_x100.nx_a2a_tdigest_subst.nx_py_quant_substrate.nxsketch_ddsketch_vs_tdigest_tail_bench.nxsketch_observability_dashboard_bench.nxsketch_tdigest_test.nxsketch_tdigest_v2_vs_v1_bench.nxsketch_tdigest_vs_materialized_quantile_bench.nx

structs

48struct Centroid {
53struct TDigest {

consts

37const NX_TD_DELTA_DEFAULT: i64 = 100
38const NX_TD_BUFFER_CAP: i64 = 1000
39const NX_TD_MAX_CENTROIDS: i64 = 2000 // bits-up scale: was 800; now supports
45const NX_TD_PI_PPB: i64 = 3141592654 // π · 10^9

functions

66func nx_tdigest_alloc(delta: i64) -> *TDigest {
84func nx_tdigest_centroid_at(td: *TDigest, i: i64) -> *Centroid {
98func nx_tdigest_w_max(td: *TDigest, cum_w: i64, total: i64) -> i64 {
called by 1: nx_tdigest_merge
128func nx_tdigest_sort_buffer(td: *TDigest) -> i64 {
called by 1: nx_tdigest_merge
157func nx_tdigest_merge(td: *TDigest) -> i64 {
271func nx_tdigest_add(td: *TDigest, value: i64) -> i64 {
286func nx_tdigest_quantile(td: *TDigest, p_milli: i64) -> i64 {
316func nx_tdigest_rank_error_ppb(delta: i64) -> i64 {
called by 1: nx_tdigest_query
323func nx_tdigest_query(td: *TDigest, p_milli: i64) -> *ApproxI64 {
334func nx_tdigest_memory_bytes(td: *TDigest) -> i64 {
called by 2: mainmain
338func nx_tdigest_n_centroids(td: *TDigest) -> i64 {
called by 2: mainmain