code wiki / (root) / nx_sketch_entropy.nx

nx_sketch_entropy.nx

buildroot/runtime/nx_sketch_entropy.nx

4539 B134 linesdepth 4pulls 4 transitivereach 0 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_entropy.nx -- streaming Shannon entropy estimator. Maintains exact frequency counts via the sovereign hash-map primitive (sketch_hash_map.nx), then computes entropy at query time: H = -Σ p_i log_2(p_i) bits = log_2(N) - (Σ f_i log_2(f_i)) / N USE CASES: - anomaly detection (sudden entropy drop = repetition spike) - change-point detection (entropy distribution shift) - feature extraction for ML over streams - DDoS detection (low entropy = single-source flood) - load-balancer fairness (high entropy = uniform distribution) EXACT WITHIN BOUNDED UNIVERSE: works for streams where distinct keys fit in the hash map (bounded by caller-chosen capacity). For unbounded universes, compose with CountSketch or SpaceSaving for approximate frequencies first. INTEGER LOG-BASE-2 (no f64): log_2(n) ≈ bitlen(n) - 1 (floor) For finer resolution: use top-bit position + fractional bits via mantissa interpolation. v1 uses floor(log_2) -- entropy under- estimates by O(1/N) per term. LOSSLESS-LANGUAGE DISCIPLINE: Returns entropy in PPM ([0, log_2(N) * 1_000_000]). ApproxI64 with NX_ENV_ABS, param_a = n_distinct (quantization bound from floor-log). Maturity = ReferenceImpl (Production when we ship the precise log-fp lookup table).

dependencies 3 imports · 0 importers

nx_syscalls.nx nx_sketch_hash_map.nx nx_sketch_types.nx nx_sketch_entropy.nx

imports: nx_syscalls.nxnx_sketch_hash_map.nxnx_sketch_types.nx

imported by: nobody (leaf or entry point)

structs

43struct StreamingEntropy

consts

none

functions

50func nx_ent_alloc(capacity: i64) -> *StreamingEntropy
62func nx_ent_add(e: *StreamingEntropy, key: i64) -> i64
74func nx_ent_log2_floor(x: i64) -> i64
called by 1: nx_ent_bits_ppm
91func nx_ent_bits_ppm(e: *StreamingEntropy) -> i64
113func nx_ent_n_distinct(e: *StreamingEntropy) -> i64
calls 1: nx_hmap_size
117func nx_ent_n_total(e: *StreamingEntropy) -> i64
123func nx_ent_query(e: *StreamingEntropy) -> *ApproxI64
132func nx_ent_memory_bytes(e: *StreamingEntropy) -> i64