nx_sketch_stream_stats.nx
buildroot/runtime/nx_sketch_stream_stats.nx
about
sketch_stream_stats.nx -- streaming mean / variance / stddev / min / max.
Fundamental single-pass streaming primitive. Maintains six i64 fields
across the stream:
count = N
sum = Σ x_i
sum_sq = Σ x_i²
min = min over stream
max = max over stream
Algebraic-identity-based estimators (NOT Welford, which is f64-stable but
loses precision when arithmetic is exact-integer):
mean = sum / count
variance = sum_sq / count - mean² (population)
stddev = isqrt(variance)
OVERFLOW BUDGET:
For sample values bounded by |x| <= V and count N:
sum: N * V — must fit i64. Safe to N * V < 2^62.
sum_sq: N * V² — same constraint applied to V² instead of V.
For V = 2^30 (1 billion): N up to ~2^32 (4 billion) safe for sum,
but sum_sq overflows at N * 2^60 -- safe to N=4.
For V = 2^20 (~1M): N up to 2^22 (~4M) safe for sum_sq.
For V = 2^16 (65K): N up to 2^30 (1B) safe.
Caller responsibility to size domain; nx_stats_safe_p returns 1 iff
adding `value` would NOT overflow.
COMPLEMENTS the sketch suite:
- sketch_reservoir: arbitrary-statistic estimation via sample-based math
- sketch_kll / sketch_tdigest: quantiles (median, p99)
- sketch_stream_stats (here): exact-integer mean/variance over the FULL
stream (no sampling, no probabilistic bounds)
LOSSLESS-LANGUAGE DISCIPLINE: nx_stats_query_mean and friends return
ApproxI64 with envelope_kind = NX_ENV_ABS, param_a = 0 (mean and
variance are EXACT under the overflow budget), conf_ppb = 1e9.
dependencies 2 imports · 2 importers
imports: nx_syscalls.nxnx_sketch_types.nx
imported by: nx_sketch_moments.nxnx_sketch_zscore.nx
structs
| 47 | struct StreamStats |
consts
| 76 | const NX_STATS_SAFE_HI: i64 = 0x2000000000000000 // 2^61 |
functions
| 58 | func nx_stats_alloc() -> *StreamStats |
| 78 | func nx_stats_safe_p(s: *StreamStats, value: i64) -> i64 called by 1: nx_stats_add |
| 90 | func nx_stats_add(s: *StreamStats, value: i64) -> i64 |
| 109 | func nx_stats_mean(s: *StreamStats) -> i64 |
| 114 | func nx_stats_variance(s: *StreamStats) -> i64 |
| 124 | func nx_stats_isqrt(x: i64) -> i64 |
| 144 | func nx_stats_stddev(s: *StreamStats) -> i64 |
| 148 | func nx_stats_min(s: *StreamStats) -> i64 |
| 152 | func nx_stats_max(s: *StreamStats) -> i64 |
| 156 | func nx_stats_count(s: *StreamStats) -> i64 |
| 164 | func nx_stats_query_mean(s: *StreamStats) -> *ApproxI64 |
| 171 | func nx_stats_query_variance(s: *StreamStats) -> *ApproxI64 |
| 182 | func nx_stats_merge(a: *StreamStats, b: *StreamStats) -> *StreamStats calls 1: nx_stats_alloc |
| 211 | func nx_stats_memory_bytes(s: *StreamStats) -> i64 called by 1: nx_zs_memory_bytes |