code wiki / (root) / sketch_tdigest_v2.nx

sketch_tdigest_v2.nx

buildroot/runtime/sketch_tdigest_v2.nx

11601 B345 linesdepth 4pulls 6 transitivereach 2 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_tdigest_v2.nx -- T-Digest with Dunning k1 arcsin-based scale. V2 vs V1 (sketch_tdigest.nx): V1 scale: w_max_norm(q) = 4·q·(1-q) · π / δ (parabolic approx) V2 scale: w_max_norm(q) = (1 - cos(2π/δ))·(1/2 - q) + sin(2π/δ)·sqrt(q(1-q)) The V2 form is the closed-form expansion of Dunning's k1(q) = (δ/2π) · arcsin(2q - 1) inverse-difference w_max = k1⁻¹(k1(q)+1) - q, via the sin-of-sum identity. At q=0.5 V1 and V2 agree; in the tails V2 is LOOSER than V1 (fewer centroids needed for the same global rank guarantee), the canonical memory-efficiency T-Digest property. V1 trades memory for tighter tail recall. V2 trades tail recall for tighter memory. Both ship -- caller picks the right tool. Per cardinal 25 ("Build Intelligence, Never Strip Features"). TRIG CONSTANTS: sin(2π/δ) and cos(2π/δ) precomputed at alloc time as Q14 fixed-point (1.0 = 16384), via Taylor series: sin(x) ≈ x - x³/6 cos(x) ≈ 1 - x²/2 + x⁴/24 For δ ∈ [10, 500] -> x ∈ [0.0126, 0.628] -> Taylor error < 1e-4 in Q14, well below the 1-LSB sketch noise floor. LOSSLESS-LANGUAGE DISCIPLINE (doc 20): Query returns ApproxI64 same NX_ENV_RANK_ERROR envelope as V1, param_a interpolates the V2-specific bound: V2 rank_err ≈ 1.5 / δ at p=0.5, ≈ 0.3 / δ at p=0.99 (tail roughly 5x looser than V1; still tighter than KLL globally).

dependencies 3 imports · 2 importers

syscalls.nx sketch_types.nx nx_vecmath.nx sketch_tdigest_v2.nx sketch_tdigest_v2_test.nx sketch_tdigest_v2_vs_v1_bench.nx

imports: syscalls.nxsketch_types.nxnx_vecmath.nx

imported by: sketch_tdigest_v2_test.nxsketch_tdigest_v2_vs_v1_bench.nx

structs

51struct CentroidV2
56struct TDigestV2

consts

38const NX_TD2_DELTA_MIN: i64 = 10
39const NX_TD2_DELTA_MAX: i64 = 500
40const NX_TD2_BUFFER_CAP: i64 = 1000
41const NX_TD2_MAX_CENTROIDS: i64 = 800
43const NX_TD2_Q14: i64 = 16384 // 1.0 fixed-point unit
44const NX_TD2_TWO_PI_Q14: i64 = 102944 // round(2π · 16384)
49const NX_TD2_SAFE_TOTAL_MAX: i64 = 100000

functions

72func nx_tdv2_alloc(delta: i64) -> *TDigestV2
106func nx_tdv2_centroid_at(td: *TDigestV2, i: i64) -> *CentroidV2
112func nx_tdv2_isqrt(x: i64) -> i64 { return vm_isqrt(x) }
121func nx_tdv2_w_max(td: *TDigestV2, cum_w: i64, total: i64) -> i64
143func nx_tdv2_sort_buffer(td: *TDigestV2) -> i64
170func nx_tdv2_merge(td: *TDigestV2) -> i64
274func nx_tdv2_add(td: *TDigestV2, value: i64) -> i64
285func nx_tdv2_quantile(td: *TDigestV2, p_milli: i64) -> i64
311func nx_tdv2_rank_error_ppb(delta: i64) -> i64
319func nx_tdv2_query(td: *TDigestV2, p_milli: i64) -> *ApproxI64
330func nx_tdv2_memory_bytes(td: *TDigestV2) -> i64
334func nx_tdv2_n_centroids(td: *TDigestV2) -> i64
339func nx_tdv2_sin_q14(td: *TDigestV2) -> i64
343func nx_tdv2_one_minus_cos_q14(td: *TDigestV2) -> i64