sketch_theta.nx
buildroot/runtime/sketch_theta.nx
about
sketch_theta.nx -- Theta sketch (Beyer/Haas 2014).
Extends KMV with EXPLICIT theta and SET INTERSECTION + DIFFERENCE.
Where KMV's theta is always implied by kth_smallest, Theta carries
theta as a first-class field so intersected sketches (which have
smaller-than-natural theta) remain valid for further composition.
Theta semantics:
theta ∈ (0, HASH_MAX]: the bound below which hashes are sampled.
Initially HASH_MAX (entire domain). After K hashes seen,
theta = kth_smallest. After intersection, theta = min of inputs'.
Cardinality estimator (theta-corrected, unbiased per Beyer 2007):
est = |hashes < theta| * HASH_MAX / theta
For a freshly-filled KMV this reduces to (K-1)*HASH_MAX/kth
which matches the KMV formula.
COMPLETES THE SET-OP TRIO:
- union: runtime/sketch_kmv.nx::nx_kmv_union
- intersection: nx_theta_intersect (THIS FILE) -- new capability
- difference: nx_theta_difference -- new capability
This is the family DataSketches ships as Theta + Tuple sketches.
We ship Theta first; Tuple (attaching auxiliary data to each hash)
is queued for v2 once the core sketches stabilize.
Reference: DataSketches Theta paper -- arXiv:1306.5664
"Beyond HyperLogLog: A Survey of Sketches for Cardinality Estimation"
dependencies 4 imports · 2 importers
imports: syscalls.nxmurmur3.nxsketch_types.nxsketch_kmv.nx
imported by: sketch_theta_intersect_vs_hll_bench.nxsketch_theta_test.nx
structs
| 38 | struct ThetaSketch { |
consts
| 36 | const NX_THETA_HASH_MAX: i64 = 4294967296 |
functions
| 48 | func nx_theta_alloc(k: i64, seed: i64) -> *ThetaSketch { |
| 67 | func nx_theta_from_kmv(kmv: *Kmv) -> *ThetaSketch {
calls 1: nx_theta_alloc |
| 84 | func nx_theta_add(t: *ThetaSketch, key: *u8, len: i64) -> i64 { |
| 141 | func nx_theta_estimate(t: *ThetaSketch) -> i64 { |
| 160 | func nx_theta_stddev_rel_ppb(k: i64) -> i64 {
called by 1: nx_theta_query |
| 170 | func nx_theta_query(t: *ThetaSketch) -> *ApproxI64 { |
| 185 | func nx_theta_min2(a: i64, b: i64) -> i64 { |
| 190 | func nx_theta_union(a: *ThetaSketch, b: *ThetaSketch) -> *ThetaSketch { |
| 269 | func nx_theta_intersect(a: *ThetaSketch, b: *ThetaSketch) -> *ThetaSketch { |
| 317 | func nx_theta_difference(a: *ThetaSketch, b: *ThetaSketch) -> *ThetaSketch { |
| 369 | func nx_theta_memory_bytes(t: *ThetaSketch) -> i64 { |