sketch_cpc_dense.nx
buildroot/runtime/sketch_cpc_dense.nx
about
sketch_cpc_dense.nx -- CPC dense bitmap mode + HIP estimator.
FIXES THE V1 SPARSE-MODE MEMORY LOSS.
Sparse mode (sketch_cpc.nx) stores explicit coupons in a hash map: ~24
bytes per distinct coupon. At 1000 coupons: ~24KB, much larger than
HLL_8's 128 bytes. That's why the v1 bench reported MEMORY: LOSES.
DENSE MODE bitmap: m * w bits total (m columns x w rows per column).
For matched accuracy with HLL_8 (m_hll=128, 9.2% stddev), CPC needs
big_M ~= 128 coupons. Configurations:
m=8, w=16 -> 128 bits = 16 bytes (8x smaller than HLL_8)
m=16, w=8 -> 128 bits = 16 bytes
m=32, w=32 -> 1024 bits = 128 bytes (matched accuracy as HLL_8x8)
HIP ESTIMATOR (Cohen 2015, Lang 2017):
On each NEW bit set (n_set transitions from k to k+1):
kappa += big_M / (big_M - k)
estimate = kappa
COMPLEMENTS sketch_cpc (sparse mode):
- sparse: small N, exact storage, larger memory per coupon
- dense: bounded memory, HIP estimator, near-optimal variance
LOSSLESS-LANGUAGE DISCIPLINE: rel_stddev ~ 1/sqrt(big_M), same as
sparse-mode CPC. Production tier (bounded memory, deterministic).
dependencies 5 imports · 4 importers
imports: syscalls.nxmurmur3.nxnx_bits.nxsketch_types.nxnx_vecmath.nx
imported by: bench_emit.nxbench_hll_vs_cpcd.nxbench_multiseed.nxsketch_cpc_dense_test.nx
structs
| 39 | struct CpcDense { |
consts
| 34 | const NX_CPCD_MIN_LG_K: i64 = 2 |
| 35 | const NX_CPCD_MAX_LG_K: i64 = 12 |
| 36 | const NX_CPCD_MIN_W: i64 = 4 |
| 37 | const NX_CPCD_MAX_W: i64 = 64 |
functions
| 54 | func nx_cpcd_clz32(x: i64) -> i64 { |
| 60 | func nx_cpcd_alloc(lg_k: i64, w: i64, seed: i64) -> *CpcDense { |
| 88 | func nx_cpcd_bit_get(c: *CpcDense, idx: i64) -> i64 { |
| 94 | func nx_cpcd_bit_set(c: *CpcDense, idx: i64) -> i64 { |
| 109 | func nx_cpcd_coupon_pos(c: *CpcDense, key: *u8, len: i64) -> i64 { |
| 120 | func nx_cpcd_add(c: *CpcDense, key: *u8, len: i64) -> i64 { |
| 135 | func nx_cpcd_estimate(c: *CpcDense) -> i64 { |
| 141 | func nx_cpcd_isqrt(x: i64) -> i64 { return vm_isqrt(x) } |
| 145 | func nx_cpcd_stddev_rel_ppb(c: *CpcDense) -> i64 { |
| 151 | func nx_cpcd_query(c: *CpcDense) -> *ApproxI64 { |
| 164 | func nx_cpcd_merge(a: *CpcDense, b: *CpcDense) -> *CpcDense { |
| 199 | func nx_cpcd_n_set(c: *CpcDense) -> i64 { return c.n_set } |
| 200 | func nx_cpcd_big_m(c: *CpcDense) -> i64 { return c.big_m } |
| 203 | func nx_cpcd_memory_bytes(c: *CpcDense) -> i64 { |