code wiki / (root) / nx_sketch_cpc_dense.nx

nx_sketch_cpc_dense.nx

buildroot/runtime/nx_sketch_cpc_dense.nx

7186 B214 linesdepth 4pulls 8 transitivereach 3 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_cpc_dense.nx -- CPC dense bitmap mode + HIP estimator. FIXES THE V1 SPARSE-MODE MEMORY LOSS. Sparse mode (sketch_cpc.nx) stores explicit coupons in a hash map: ~24 bytes per distinct coupon. At 1000 coupons: ~24KB, much larger than HLL_8's 128 bytes. That's why the v1 bench reported MEMORY: LOSES. DENSE MODE bitmap: m * w bits total (m columns x w rows per column). For matched accuracy with HLL_8 (m_hll=128, 9.2% stddev), CPC needs big_M ~= 128 coupons. Configurations: m=8, w=16 -> 128 bits = 16 bytes (8x smaller than HLL_8) m=16, w=8 -> 128 bits = 16 bytes m=32, w=32 -> 1024 bits = 128 bytes (matched accuracy as HLL_8x8) HIP ESTIMATOR (Cohen 2015, Lang 2017): On each NEW bit set (n_set transitions from k to k+1): kappa += big_M / (big_M - k) estimate = kappa COMPLEMENTS sketch_cpc (sparse mode): - sparse: small N, exact storage, larger memory per coupon - dense: bounded memory, HIP estimator, near-optimal variance LOSSLESS-LANGUAGE DISCIPLINE: rel_stddev ~ 1/sqrt(big_M), same as sparse-mode CPC. Production tier (bounded memory, deterministic).

dependencies 5 imports · 3 importers

nx_syscalls.nx nx_murmur3.nx nx_bits.nx nx_sketch_types.nx nx_vecmath.nx nx_sketch_cpc_dense.nx nx_bench_emit.nx nx_bench_hll_vs_cpcd.nx nx_bench_multiseed.nx

imports: nx_syscalls.nxnx_murmur3.nxnx_bits.nxnx_sketch_types.nxnx_vecmath.nx

imported by: nx_bench_emit.nxnx_bench_hll_vs_cpcd.nxnx_bench_multiseed.nx

structs

48struct CpcDense

consts

39const NX_MAGIC_1000000: i64 = 1000000
40const NX_MAGIC_1000000000: i64 = 1000000000
41const NX_MAGIC_682700000: i64 = 682700000
43const NX_CPCD_MIN_LG_K: i64 = 2
44const NX_CPCD_MAX_LG_K: i64 = 12
45const NX_CPCD_MIN_W: i64 = 4
46const NX_CPCD_MAX_W: i64 = 64

functions

63func nx_cpcd_clz32(x: i64) -> i64
called by 1: nx_cpcd_coupon_pos calls 1: nx_bits_clz32
69func nx_cpcd_alloc(lg_k: i64, w: i64, seed: i64) -> *CpcDense
called by 7: mainmainmainmainmainmain+1 calls 1: sys_mmap
97func nx_cpcd_bit_get(c: *CpcDense, idx: i64) -> i64
103func nx_cpcd_bit_set(c: *CpcDense, idx: i64) -> i64
118func nx_cpcd_coupon_pos(c: *CpcDense, key: *u8, len: i64) -> i64
called by 1: nx_cpcd_add calls 2: murmur3_32nx_cpcd_clz32
129func nx_cpcd_add(c: *CpcDense, key: *u8, len: i64) -> i64
144func nx_cpcd_estimate(c: *CpcDense) -> i64
150func nx_cpcd_isqrt(x: i64) -> i64 { return vm_isqrt(x) }
called by 1: nx_cpcd_stddev_rel_ppb calls 1: vm_isqrt
154func nx_cpcd_stddev_rel_ppb(c: *CpcDense) -> i64
called by 1: nx_cpcd_query calls 1: nx_cpcd_isqrt
160func nx_cpcd_query(c: *CpcDense) -> *ApproxI64
173func nx_cpcd_merge(a: *CpcDense, b: *CpcDense) -> *CpcDense
208func nx_cpcd_n_set(c: *CpcDense) -> i64 { return c.n_set }
209func nx_cpcd_big_m(c: *CpcDense) -> i64 { return c.big_m }
212func nx_cpcd_memory_bytes(c: *CpcDense) -> i64
called by 4: mainmainmainmain