sketch_kmv.nx
buildroot/runtime/sketch_kmv.nx
about
sketch_kmv.nx -- KMV (K-Minimum-Values) sketch.
Bar-Yossef et al. 2002 + Beyer 2007. Maintain the K smallest
32-bit hash values seen. Direct ancestor of DataSketches'
Theta sketches; we ship the foundation primitive first then
build Theta on top.
CAPABILITIES (set-operation cardinality):
- cardinality: (K-1) * 2^32 / kth_smallest when |set| > K
total_inserted_unique when |set| <= K
- union: K smallest across A.values ∪ B.values
- Jaccard: |A.kmins ∩ B.kmins| / |A.kmins ∪ B.kmins|
unbiased estimator of true Jaccard
Error: rel_stddev ~ 1/sqrt(K-1) at 1-sigma.
For K=4096: 1.56% stddev. For K=16384: 0.78%.
COMPLEMENTS HLL:
- HLL: better space per accuracy point for pure cardinality.
- KMV: SAME sketch supports union AND intersection (Jaccard).
The trade-off is well-studied; both ship in DataSketches
(HLL.java and Theta.java) -- we ship the same axis pair.
Hash domain: 32-bit (murmur3_32). We store hashes as i64 to
keep arithmetic simple; the value space is [0, 2^32). Hash 0
is mapped to 1 (matches Cuckoo's empty-sentinel pattern; KMV
uses the special "all-slots-filled-with-max" sentinel via
the n_items counter instead, but mapping 0->1 avoids edge
cases in the cardinality formula).
dependencies 3 imports · 3 importers
imports: syscalls.nxmurmur3.nxsketch_types.nx
imported by: sketch_kmv_jaccard_vs_naive_bench.nxsketch_kmv_test.nxsketch_theta.nx
structs
| 41 | struct Kmv { |
consts
| 35 | const NX_KMV_K_MIN: i64 = 16 |
| 36 | const NX_KMV_K_MAX: i64 = 65536 |
| 39 | const NX_KMV_HASH_MAX: i64 = 4294967296 |
functions
| 50 | func nx_kmv_alloc(k: i64, seed: i64) -> *Kmv { |
| 65 | func nx_kmv_hash(kmv: *Kmv, key: *u8, len: i64) -> i64 { |
| 78 | func nx_kmv_add(kmv: *Kmv, key: *u8, len: i64) -> i64 { |
| 118 | func nx_kmv_add_hash(kmv: *Kmv, raw_h: i64) -> i64 {
called by 1: main |
| 161 | func nx_kmv_estimate(kmv: *Kmv) -> i64 { |
| 172 | func nx_kmv_stddev_rel_ppb(k: i64) -> i64 {
called by 1: nx_kmv_query |
| 182 | func nx_kmv_query(kmv: *Kmv) -> *ApproxI64 { |
| 198 | func nx_kmv_union(a: *Kmv, b: *Kmv) -> *Kmv { |
| 262 | func nx_kmv_jaccard_ppm(a: *Kmv, b: *Kmv) -> i64 { |
| 308 | func nx_kmv_memory_bytes(kmv: *Kmv) -> i64 {
called by 1: main |