code wiki / (root) / sketch_stream_stats.nx

sketch_stream_stats.nx

buildroot/runtime/sketch_stream_stats.nx

6178 B190 linesdepth 4pulls 6 transitivereach 5 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_stream_stats.nx -- streaming mean / variance / stddev / min / max. Fundamental single-pass streaming primitive. Maintains six i64 fields across the stream: count = N sum = Σ x_i sum_sq = Σ x_i² min = min over stream max = max over stream Algebraic-identity-based estimators (NOT Welford, which is f64-stable but loses precision when arithmetic is exact-integer): mean = sum / count variance = sum_sq / count - mean² (population) stddev = isqrt(variance) OVERFLOW BUDGET: For sample values bounded by |x| <= V and count N: sum: N * V — must fit i64. Safe to N * V < 2^62. sum_sq: N * V² — same constraint applied to V² instead of V. For V = 2^30 (1 billion): N up to ~2^32 (4 billion) safe for sum, but sum_sq overflows at N * 2^60 -- safe to N=4. For V = 2^20 (~1M): N up to 2^22 (~4M) safe for sum_sq. For V = 2^16 (65K): N up to 2^30 (1B) safe. Caller responsibility to size domain; nx_stats_safe_p returns 1 iff adding `value` would NOT overflow. COMPLEMENTS the sketch suite: - sketch_reservoir: arbitrary-statistic estimation via sample-based math - sketch_kll / sketch_tdigest: quantiles (median, p99) - sketch_stream_stats (here): exact-integer mean/variance over the FULL stream (no sampling, no probabilistic bounds) LOSSLESS-LANGUAGE DISCIPLINE: nx_stats_query_mean and friends return ApproxI64 with envelope_kind = NX_ENV_ABS, param_a = 0 (mean and variance are EXACT under the overflow budget), conf_ppb = 1e9.

dependencies 3 imports · 4 importers

syscalls.nx sketch_types.nx nx_vecmath.nx sketch_stream_stats.nx sketch_moments.nx sketch_stream_stats_test.nx sketch_zscore.nx sketch_zscore_test.nx

imports: syscalls.nxsketch_types.nxnx_vecmath.nx

imported by: sketch_moments.nxsketch_stream_stats_test.nxsketch_zscore.nxsketch_zscore_test.nx

structs

42struct StreamStats {

consts

71const NX_STATS_SAFE_HI: i64 = 0x2000000000000000 // 2^61

functions

53func nx_stats_alloc() -> *StreamStats {
73func nx_stats_safe_p(s: *StreamStats, value: i64) -> i64 {
85func nx_stats_add(s: *StreamStats, value: i64) -> i64 {
104func nx_stats_mean(s: *StreamStats) -> i64 {
109func nx_stats_variance(s: *StreamStats) -> i64 {
119func nx_stats_isqrt(x: i64) -> i64 { return vm_isqrt(x) }
121func nx_stats_stddev(s: *StreamStats) -> i64 {
125func nx_stats_min(s: *StreamStats) -> i64 {
129func nx_stats_max(s: *StreamStats) -> i64 {
133func nx_stats_count(s: *StreamStats) -> i64 {
141func nx_stats_query_mean(s: *StreamStats) -> *ApproxI64 {
148func nx_stats_query_variance(s: *StreamStats) -> *ApproxI64 {
159func nx_stats_merge(a: *StreamStats, b: *StreamStats) -> *StreamStats {
188func nx_stats_memory_bytes(s: *StreamStats) -> i64 {