code wiki / (root) / sketch_cpc_dense.nx

sketch_cpc_dense.nx

buildroot/runtime/sketch_cpc_dense.nx

7252 B219 linesdepth 4pulls 6 transitivereach 4 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_cpc_dense.nx -- CPC dense bitmap mode + HIP estimator. FIXES THE V1 SPARSE-MODE MEMORY LOSS. Sparse mode (sketch_cpc.nx) stores explicit coupons in a hash map: ~24 bytes per distinct coupon. At 1000 coupons: ~24KB, much larger than HLL_8's 128 bytes. That's why the v1 bench reported MEMORY: LOSES. DENSE MODE bitmap: m * w bits total (m columns x w rows per column). For matched accuracy with HLL_8 (m_hll=128, 9.2% stddev), CPC needs big_M ~= 128 coupons. Configurations: m=8, w=16 -> 128 bits = 16 bytes (8x smaller than HLL_8) m=16, w=8 -> 128 bits = 16 bytes m=32, w=32 -> 1024 bits = 128 bytes (matched accuracy as HLL_8x8) HIP ESTIMATOR (Cohen 2015, Lang 2017): On each NEW bit set (n_set transitions from k to k+1): kappa += big_M / (big_M - k) estimate = kappa COMPLEMENTS sketch_cpc (sparse mode): - sparse: small N, exact storage, larger memory per coupon - dense: bounded memory, HIP estimator, near-optimal variance LOSSLESS-LANGUAGE DISCIPLINE: rel_stddev ~ 1/sqrt(big_M), same as sparse-mode CPC. Production tier (bounded memory, deterministic).

dependencies 4 imports · 4 importers

syscalls.nx murmur3.nx nx_bits.nx sketch_types.nx sketch_cpc_dense.nx bench_emit.nx bench_hll_vs_cpcd.nx bench_multiseed.nx sketch_cpc_dense_test.nx

imports: syscalls.nxmurmur3.nxnx_bits.nxsketch_types.nx

imported by: bench_emit.nxbench_hll_vs_cpcd.nxbench_multiseed.nxsketch_cpc_dense_test.nx

structs

38struct CpcDense {

consts

33const NX_CPCD_MIN_LG_K: i64 = 2
34const NX_CPCD_MAX_LG_K: i64 = 12
35const NX_CPCD_MIN_W: i64 = 4
36const NX_CPCD_MAX_W: i64 = 64

functions

53func nx_cpcd_clz32(x: i64) -> i64 {
called by 1: nx_cpcd_coupon_pos calls 1: nx_bits_clz32
59func nx_cpcd_alloc(lg_k: i64, w: i64, seed: i64) -> *CpcDense {
87func nx_cpcd_bit_get(c: *CpcDense, idx: i64) -> i64 {
93func nx_cpcd_bit_set(c: *CpcDense, idx: i64) -> i64 {
108func nx_cpcd_coupon_pos(c: *CpcDense, key: *u8, len: i64) -> i64 {
called by 1: nx_cpcd_add calls 2: murmur3_32nx_cpcd_clz32
119func nx_cpcd_add(c: *CpcDense, key: *u8, len: i64) -> i64 {
134func nx_cpcd_estimate(c: *CpcDense) -> i64 {
140func nx_cpcd_isqrt(x: i64) -> i64 {
159func nx_cpcd_stddev_rel_ppb(c: *CpcDense) -> i64 {
called by 1: nx_cpcd_query calls 1: nx_cpcd_isqrt
165func nx_cpcd_query(c: *CpcDense) -> *ApproxI64 {
178func nx_cpcd_merge(a: *CpcDense, b: *CpcDense) -> *CpcDense {
213func nx_cpcd_n_set(c: *CpcDense) -> i64 { return c.n_set }
called by 1: main
214func nx_cpcd_big_m(c: *CpcDense) -> i64 { return c.big_m }
called by 1: main
217func nx_cpcd_memory_bytes(c: *CpcDense) -> i64 {
called by 3: mainmainmain