nx_sketch_cpc_dense.nx
buildroot/runtime/nx_sketch_cpc_dense.nx
about
sketch_cpc_dense.nx -- CPC dense bitmap mode + HIP estimator.
FIXES THE V1 SPARSE-MODE MEMORY LOSS.
Sparse mode (sketch_cpc.nx) stores explicit coupons in a hash map: ~24
bytes per distinct coupon. At 1000 coupons: ~24KB, much larger than
HLL_8's 128 bytes. That's why the v1 bench reported MEMORY: LOSES.
DENSE MODE bitmap: m * w bits total (m columns x w rows per column).
For matched accuracy with HLL_8 (m_hll=128, 9.2% stddev), CPC needs
big_M ~= 128 coupons. Configurations:
m=8, w=16 -> 128 bits = 16 bytes (8x smaller than HLL_8)
m=16, w=8 -> 128 bits = 16 bytes
m=32, w=32 -> 1024 bits = 128 bytes (matched accuracy as HLL_8x8)
HIP ESTIMATOR (Cohen 2015, Lang 2017):
On each NEW bit set (n_set transitions from k to k+1):
kappa += big_M / (big_M - k)
estimate = kappa
COMPLEMENTS sketch_cpc (sparse mode):
- sparse: small N, exact storage, larger memory per coupon
- dense: bounded memory, HIP estimator, near-optimal variance
LOSSLESS-LANGUAGE DISCIPLINE: rel_stddev ~ 1/sqrt(big_M), same as
sparse-mode CPC. Production tier (bounded memory, deterministic).
dependencies 4 imports · 3 importers
imports: nx_syscalls.nxnx_murmur3.nxnx_bits.nxnx_sketch_types.nx
imported by: nx_bench_emit.nxnx_bench_hll_vs_cpcd.nxnx_bench_multiseed.nx
structs
| 47 | struct CpcDense |
consts
| 38 | const NX_MAGIC_1000000: i64 = 1000000 |
| 39 | const NX_MAGIC_1000000000: i64 = 1000000000 |
| 40 | const NX_MAGIC_682700000: i64 = 682700000 |
| 42 | const NX_CPCD_MIN_LG_K: i64 = 2 |
| 43 | const NX_CPCD_MAX_LG_K: i64 = 12 |
| 44 | const NX_CPCD_MIN_W: i64 = 4 |
| 45 | const NX_CPCD_MAX_W: i64 = 64 |
functions
| 62 | func nx_cpcd_clz32(x: i64) -> i64 |
| 68 | func nx_cpcd_alloc(lg_k: i64, w: i64, seed: i64) -> *CpcDense |
| 96 | func nx_cpcd_bit_get(c: *CpcDense, idx: i64) -> i64 |
| 102 | func nx_cpcd_bit_set(c: *CpcDense, idx: i64) -> i64 |
| 117 | func nx_cpcd_coupon_pos(c: *CpcDense, key: *u8, len: i64) -> i64 |
| 128 | func nx_cpcd_add(c: *CpcDense, key: *u8, len: i64) -> i64 |
| 143 | func nx_cpcd_estimate(c: *CpcDense) -> i64 |
| 149 | func nx_cpcd_isqrt(x: i64) -> i64 called by 1: nx_cpcd_stddev_rel_ppb |
| 168 | func nx_cpcd_stddev_rel_ppb(c: *CpcDense) -> i64 |
| 174 | func nx_cpcd_query(c: *CpcDense) -> *ApproxI64 |
| 187 | func nx_cpcd_merge(a: *CpcDense, b: *CpcDense) -> *CpcDense |
| 222 | func nx_cpcd_n_set(c: *CpcDense) -> i64 { return c.n_set } |
| 223 | func nx_cpcd_big_m(c: *CpcDense) -> i64 { return c.big_m } |
| 226 | func nx_cpcd_memory_bytes(c: *CpcDense) -> i64 |