code wiki / (root) / nx_sketch_kmv.nx

nx_sketch_kmv.nx

buildroot/runtime/nx_sketch_kmv.nx

10241 B316 linesdepth 3pulls 4 transitivereach 1 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_kmv.nx -- KMV (K-Minimum-Values) sketch. Bar-Yossef et al. 2002 + Beyer 2007. Maintain the K smallest 32-bit hash values seen. Direct ancestor of DataSketches' Theta sketches; we ship the foundation primitive first then build Theta on top. CAPABILITIES (set-operation cardinality): - cardinality: (K-1) * 2^32 / kth_smallest when |set| > K total_inserted_unique when |set| <= K - union: K smallest across A.values ∪ B.values - Jaccard: |A.kmins ∩ B.kmins| / |A.kmins ∪ B.kmins| unbiased estimator of true Jaccard Error: rel_stddev ~ 1/sqrt(K-1) at 1-sigma. For K=4096: 1.56% stddev. For K=16384: 0.78%. COMPLEMENTS HLL: - HLL: better space per accuracy point for pure cardinality. - KMV: SAME sketch supports union AND intersection (Jaccard). The trade-off is well-studied; both ship in DataSketches (HLL.java and Theta.java) -- we ship the same axis pair. Hash domain: 32-bit (murmur3_32). We store hashes as i64 to keep arithmetic simple; the value space is [0, 2^32). Hash 0 is mapped to 1 (matches Cuckoo's empty-sentinel pattern; KMV uses the special "all-slots-filled-with-max" sentinel via the n_items counter instead, but mapping 0->1 avoids edge cases in the cardinality formula).

dependencies 3 imports · 1 importers

nx_syscalls.nx nx_murmur3.nx nx_sketch_types.nx nx_sketch_kmv.nx nx_sketch_theta.nx

imports: nx_syscalls.nxnx_murmur3.nxnx_sketch_types.nx

imported by: nx_sketch_theta.nx

structs

47struct Kmv

consts

41const NX_KMV_K_MIN: i64 = 16
42const NX_KMV_K_MAX: i64 = 65536
45const NX_KMV_HASH_MAX: i64 = 4294967296

functions

56func nx_kmv_alloc(k: i64, seed: i64) -> *Kmv
called by 1: nx_kmv_union calls 1: sys_mmap
71func nx_kmv_hash(kmv: *Kmv, key: *u8, len: i64) -> i64
called by 1: nx_kmv_add calls 1: murmur3_32
84func nx_kmv_add(kmv: *Kmv, key: *u8, len: i64) -> i64
calls 1: nx_kmv_hash
124func nx_kmv_add_hash(kmv: *Kmv, raw_h: i64) -> i64
167func nx_kmv_estimate(kmv: *Kmv) -> i64
called by 1: nx_kmv_query
178func nx_kmv_stddev_rel_ppb(k: i64) -> i64
called by 1: nx_kmv_query
188func nx_kmv_query(kmv: *Kmv) -> *ApproxI64
204func nx_kmv_union(a: *Kmv, b: *Kmv) -> *Kmv
calls 1: nx_kmv_alloc
268func nx_kmv_jaccard_ppm(a: *Kmv, b: *Kmv) -> i64
314func nx_kmv_memory_bytes(kmv: *Kmv) -> i64