sketch_stream_stats.nx
buildroot/runtime/sketch_stream_stats.nx
about
sketch_stream_stats.nx -- streaming mean / variance / stddev / min / max.
Fundamental single-pass streaming primitive. Maintains six i64 fields
across the stream:
count = N
sum = Σ x_i
sum_sq = Σ x_i²
min = min over stream
max = max over stream
Algebraic-identity-based estimators (NOT Welford, which is f64-stable but
loses precision when arithmetic is exact-integer):
mean = sum / count
variance = sum_sq / count - mean² (population)
stddev = isqrt(variance)
OVERFLOW BUDGET:
For sample values bounded by |x| <= V and count N:
sum: N * V — must fit i64. Safe to N * V < 2^62.
sum_sq: N * V² — same constraint applied to V² instead of V.
For V = 2^30 (1 billion): N up to ~2^32 (4 billion) safe for sum,
but sum_sq overflows at N * 2^60 -- safe to N=4.
For V = 2^20 (~1M): N up to 2^22 (~4M) safe for sum_sq.
For V = 2^16 (65K): N up to 2^30 (1B) safe.
Caller responsibility to size domain; nx_stats_safe_p returns 1 iff
adding `value` would NOT overflow.
COMPLEMENTS the sketch suite:
- sketch_reservoir: arbitrary-statistic estimation via sample-based math
- sketch_kll / sketch_tdigest: quantiles (median, p99)
- sketch_stream_stats (here): exact-integer mean/variance over the FULL
stream (no sampling, no probabilistic bounds)
LOSSLESS-LANGUAGE DISCIPLINE: nx_stats_query_mean and friends return
ApproxI64 with envelope_kind = NX_ENV_ABS, param_a = 0 (mean and
variance are EXACT under the overflow budget), conf_ppb = 1e9.
dependencies 2 imports · 4 importers
imports: syscalls.nxsketch_types.nx
imported by: sketch_moments.nxsketch_stream_stats_test.nxsketch_zscore.nxsketch_zscore_test.nx
structs
| 41 | struct StreamStats { |
consts
| 70 | const NX_STATS_SAFE_HI: i64 = 0x2000000000000000 // 2^61 |
functions
| 52 | func nx_stats_alloc() -> *StreamStats { |
| 72 | func nx_stats_safe_p(s: *StreamStats, value: i64) -> i64 { |
| 84 | func nx_stats_add(s: *StreamStats, value: i64) -> i64 { |
| 103 | func nx_stats_mean(s: *StreamStats) -> i64 { |
| 108 | func nx_stats_variance(s: *StreamStats) -> i64 { |
| 118 | func nx_stats_isqrt(x: i64) -> i64 { |
| 138 | func nx_stats_stddev(s: *StreamStats) -> i64 { |
| 142 | func nx_stats_min(s: *StreamStats) -> i64 {
called by 1: main |
| 146 | func nx_stats_max(s: *StreamStats) -> i64 {
called by 1: main |
| 150 | func nx_stats_count(s: *StreamStats) -> i64 { |
| 158 | func nx_stats_query_mean(s: *StreamStats) -> *ApproxI64 { |
| 165 | func nx_stats_query_variance(s: *StreamStats) -> *ApproxI64 { |
| 176 | func nx_stats_merge(a: *StreamStats, b: *StreamStats) -> *StreamStats { |
| 205 | func nx_stats_memory_bytes(s: *StreamStats) -> i64 {
called by 1: nx_zs_memory_bytes |