code wiki / (root) / sketch_tdigest_v2.nx

sketch_tdigest_v2.nx

buildroot/runtime/sketch_tdigest_v2.nx

11895 B359 linesdepth 4pulls 4 transitivereach 2 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_tdigest_v2.nx -- T-Digest with Dunning k1 arcsin-based scale. V2 vs V1 (sketch_tdigest.nx): V1 scale: w_max_norm(q) = 4·q·(1-q) · π / δ (parabolic approx) V2 scale: w_max_norm(q) = (1 - cos(2π/δ))·(1/2 - q) + sin(2π/δ)·sqrt(q(1-q)) The V2 form is the closed-form expansion of Dunning's k1(q) = (δ/2π) · arcsin(2q - 1) inverse-difference w_max = k1⁻¹(k1(q)+1) - q, via the sin-of-sum identity. At q=0.5 V1 and V2 agree; in the tails V2 is LOOSER than V1 (fewer centroids needed for the same global rank guarantee), the canonical memory-efficiency T-Digest property. V1 trades memory for tighter tail recall. V2 trades tail recall for tighter memory. Both ship -- caller picks the right tool. Per cardinal 25 ("Build Intelligence, Never Strip Features"). TRIG CONSTANTS: sin(2π/δ) and cos(2π/δ) precomputed at alloc time as Q14 fixed-point (1.0 = 16384), via Taylor series: sin(x) ≈ x - x³/6 cos(x) ≈ 1 - x²/2 + x⁴/24 For δ ∈ [10, 500] -> x ∈ [0.0126, 0.628] -> Taylor error < 1e-4 in Q14, well below the 1-LSB sketch noise floor. LOSSLESS-LANGUAGE DISCIPLINE (doc 20): Query returns ApproxI64 same NX_ENV_RANK_ERROR envelope as V1, param_a interpolates the V2-specific bound: V2 rank_err ≈ 1.5 / δ at p=0.5, ≈ 0.3 / δ at p=0.99 (tail roughly 5x looser than V1; still tighter than KLL globally).

dependencies 2 imports · 2 importers

syscalls.nx sketch_types.nx sketch_tdigest_v2.nx sketch_tdigest_v2_test.nx sketch_tdigest_v2_vs_v1_bench.nx

imports: syscalls.nxsketch_types.nx

imported by: sketch_tdigest_v2_test.nxsketch_tdigest_v2_vs_v1_bench.nx

structs

50struct CentroidV2
55struct TDigestV2

consts

37const NX_TD2_DELTA_MIN: i64 = 10
38const NX_TD2_DELTA_MAX: i64 = 500
39const NX_TD2_BUFFER_CAP: i64 = 1000
40const NX_TD2_MAX_CENTROIDS: i64 = 800
42const NX_TD2_Q14: i64 = 16384 // 1.0 fixed-point unit
43const NX_TD2_TWO_PI_Q14: i64 = 102944 // round(2π · 16384)
48const NX_TD2_SAFE_TOTAL_MAX: i64 = 100000

functions

71func nx_tdv2_alloc(delta: i64) -> *TDigestV2
called by 2: mainmain
105func nx_tdv2_centroid_at(td: *TDigestV2, i: i64) -> *CentroidV2
111func nx_tdv2_isqrt(x: i64) -> i64
called by 1: nx_tdv2_w_max
135func nx_tdv2_w_max(td: *TDigestV2, cum_w: i64, total: i64) -> i64
called by 1: nx_tdv2_merge calls 1: nx_tdv2_isqrt
157func nx_tdv2_sort_buffer(td: *TDigestV2) -> i64
called by 1: nx_tdv2_merge
184func nx_tdv2_merge(td: *TDigestV2) -> i64
288func nx_tdv2_add(td: *TDigestV2, value: i64) -> i64
called by 2: mainmain calls 1: nx_tdv2_merge
299func nx_tdv2_quantile(td: *TDigestV2, p_milli: i64) -> i64
325func nx_tdv2_rank_error_ppb(delta: i64) -> i64
called by 1: nx_tdv2_query
333func nx_tdv2_query(td: *TDigestV2, p_milli: i64) -> *ApproxI64
344func nx_tdv2_memory_bytes(td: *TDigestV2) -> i64
348func nx_tdv2_n_centroids(td: *TDigestV2) -> i64
called by 2: mainmain
353func nx_tdv2_sin_q14(td: *TDigestV2) -> i64
called by 1: main
357func nx_tdv2_one_minus_cos_q14(td: *TDigestV2) -> i64
called by 1: main