nx_sketch_entropy.nx
buildroot/runtime/nx_sketch_entropy.nx
about
sketch_entropy.nx -- streaming Shannon entropy estimator.
Maintains exact frequency counts via the sovereign hash-map primitive
(sketch_hash_map.nx), then computes entropy at query time:
H = -Σ p_i log_2(p_i) bits
= log_2(N) - (Σ f_i log_2(f_i)) / N
USE CASES:
- anomaly detection (sudden entropy drop = repetition spike)
- change-point detection (entropy distribution shift)
- feature extraction for ML over streams
- DDoS detection (low entropy = single-source flood)
- load-balancer fairness (high entropy = uniform distribution)
EXACT WITHIN BOUNDED UNIVERSE: works for streams where distinct
keys fit in the hash map (bounded by caller-chosen capacity).
For unbounded universes, compose with CountSketch or SpaceSaving
for approximate frequencies first.
INTEGER LOG-BASE-2 (no f64):
log_2(n) ≈ bitlen(n) - 1 (floor)
For finer resolution: use top-bit position + fractional bits via
mantissa interpolation. v1 uses floor(log_2) -- entropy under-
estimates by O(1/N) per term.
LOSSLESS-LANGUAGE DISCIPLINE:
Returns entropy in PPM ([0, log_2(N) * 1_000_000]).
ApproxI64 with NX_ENV_ABS, param_a = n_distinct (quantization
bound from floor-log). Maturity = ReferenceImpl (Production
when we ship the precise log-fp lookup table).
dependencies 3 imports · 0 importers
imports: nx_syscalls.nxnx_sketch_hash_map.nxnx_sketch_types.nx
imported by: nobody (leaf or entry point)
structs
| 43 | struct StreamingEntropy |
consts
| none |
functions
| 50 | func nx_ent_alloc(capacity: i64) -> *StreamingEntropy |
| 62 | func nx_ent_add(e: *StreamingEntropy, key: i64) -> i64 |
| 74 | func nx_ent_log2_floor(x: i64) -> i64 called by 1: nx_ent_bits_ppm |
| 91 | func nx_ent_bits_ppm(e: *StreamingEntropy) -> i64 |
| 113 | func nx_ent_n_distinct(e: *StreamingEntropy) -> i64 calls 1: nx_hmap_size |
| 117 | func nx_ent_n_total(e: *StreamingEntropy) -> i64 |
| 123 | func nx_ent_query(e: *StreamingEntropy) -> *ApproxI64 |
| 132 | func nx_ent_memory_bytes(e: *StreamingEntropy) -> i64 calls 1: nx_hmap_memory_bytes |