code wiki / (root) / sketch_theta.nx

sketch_theta.nx

buildroot/runtime/sketch_theta.nx

12861 B371 linesdepth 5pulls 6 transitivereach 2 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_theta.nx -- Theta sketch (Beyer/Haas 2014). Extends KMV with EXPLICIT theta and SET INTERSECTION + DIFFERENCE. Where KMV's theta is always implied by kth_smallest, Theta carries theta as a first-class field so intersected sketches (which have smaller-than-natural theta) remain valid for further composition. Theta semantics: theta ∈ (0, HASH_MAX]: the bound below which hashes are sampled. Initially HASH_MAX (entire domain). After K hashes seen, theta = kth_smallest. After intersection, theta = min of inputs'. Cardinality estimator (theta-corrected, unbiased per Beyer 2007): est = |hashes < theta| * HASH_MAX / theta For a freshly-filled KMV this reduces to (K-1)*HASH_MAX/kth which matches the KMV formula. COMPLETES THE SET-OP TRIO: - union: runtime/sketch_kmv.nx::nx_kmv_union - intersection: nx_theta_intersect (THIS FILE) -- new capability - difference: nx_theta_difference -- new capability This is the family DataSketches ships as Theta + Tuple sketches. We ship Theta first; Tuple (attaching auxiliary data to each hash) is queued for v2 once the core sketches stabilize. Reference: DataSketches Theta paper -- arXiv:1306.5664 "Beyond HyperLogLog: A Survey of Sketches for Cardinality Estimation"

dependencies 4 imports · 2 importers

syscalls.nx murmur3.nx sketch_types.nx sketch_kmv.nx sketch_theta.nx sketch_theta_intersect_vs_hll_benc sketch_theta_test.nx

imports: syscalls.nxmurmur3.nxsketch_types.nxsketch_kmv.nx

imported by: sketch_theta_intersect_vs_hll_bench.nxsketch_theta_test.nx

structs

38struct ThetaSketch {

consts

36const NX_THETA_HASH_MAX: i64 = 4294967296

functions

48func nx_theta_alloc(k: i64, seed: i64) -> *ThetaSketch {
67func nx_theta_from_kmv(kmv: *Kmv) -> *ThetaSketch {
calls 1: nx_theta_alloc
84func nx_theta_add(t: *ThetaSketch, key: *u8, len: i64) -> i64 {
called by 2: mainmain calls 1: murmur3_32
141func nx_theta_estimate(t: *ThetaSketch) -> i64 {
160func nx_theta_stddev_rel_ppb(k: i64) -> i64 {
called by 1: nx_theta_query
170func nx_theta_query(t: *ThetaSketch) -> *ApproxI64 {
185func nx_theta_min2(a: i64, b: i64) -> i64 {
190func nx_theta_union(a: *ThetaSketch, b: *ThetaSketch) -> *ThetaSketch {
269func nx_theta_intersect(a: *ThetaSketch, b: *ThetaSketch) -> *ThetaSketch {
317func nx_theta_difference(a: *ThetaSketch, b: *ThetaSketch) -> *ThetaSketch {
called by 1: main calls 2: nx_theta_allocnx_theta_min2
369func nx_theta_memory_bytes(t: *ThetaSketch) -> i64 {