code wiki / (root) / nx_sketch_cpc_dense.nx

nx_sketch_cpc_dense.nx

buildroot/runtime/nx_sketch_cpc_dense.nx

7480 B228 linesdepth 4pulls 6 transitivereach 3 importersview sourcekind sketch/demotopic sketch
docsdependenciesstructsconstsfunctions

about

sketch_cpc_dense.nx -- CPC dense bitmap mode + HIP estimator. FIXES THE V1 SPARSE-MODE MEMORY LOSS. Sparse mode (sketch_cpc.nx) stores explicit coupons in a hash map: ~24 bytes per distinct coupon. At 1000 coupons: ~24KB, much larger than HLL_8's 128 bytes. That's why the v1 bench reported MEMORY: LOSES. DENSE MODE bitmap: m * w bits total (m columns x w rows per column). For matched accuracy with HLL_8 (m_hll=128, 9.2% stddev), CPC needs big_M ~= 128 coupons. Configurations: m=8, w=16 -> 128 bits = 16 bytes (8x smaller than HLL_8) m=16, w=8 -> 128 bits = 16 bytes m=32, w=32 -> 1024 bits = 128 bytes (matched accuracy as HLL_8x8) HIP ESTIMATOR (Cohen 2015, Lang 2017): On each NEW bit set (n_set transitions from k to k+1): kappa += big_M / (big_M - k) estimate = kappa COMPLEMENTS sketch_cpc (sparse mode): - sparse: small N, exact storage, larger memory per coupon - dense: bounded memory, HIP estimator, near-optimal variance LOSSLESS-LANGUAGE DISCIPLINE: rel_stddev ~ 1/sqrt(big_M), same as sparse-mode CPC. Production tier (bounded memory, deterministic).

dependencies 4 imports · 3 importers

nx_syscalls.nx nx_murmur3.nx nx_bits.nx nx_sketch_types.nx nx_sketch_cpc_dense.nx nx_bench_emit.nx nx_bench_hll_vs_cpcd.nx nx_bench_multiseed.nx

imports: nx_syscalls.nxnx_murmur3.nxnx_bits.nxnx_sketch_types.nx

imported by: nx_bench_emit.nxnx_bench_hll_vs_cpcd.nxnx_bench_multiseed.nx

structs

47struct CpcDense

consts

38const NX_MAGIC_1000000: i64 = 1000000
39const NX_MAGIC_1000000000: i64 = 1000000000
40const NX_MAGIC_682700000: i64 = 682700000
42const NX_CPCD_MIN_LG_K: i64 = 2
43const NX_CPCD_MAX_LG_K: i64 = 12
44const NX_CPCD_MIN_W: i64 = 4
45const NX_CPCD_MAX_W: i64 = 64

functions

62func nx_cpcd_clz32(x: i64) -> i64
called by 1: nx_cpcd_coupon_pos calls 1: nx_bits_clz32
68func nx_cpcd_alloc(lg_k: i64, w: i64, seed: i64) -> *CpcDense
called by 4: mainmainmainnx_cpcd_merge calls 1: sys_mmap
96func nx_cpcd_bit_get(c: *CpcDense, idx: i64) -> i64
102func nx_cpcd_bit_set(c: *CpcDense, idx: i64) -> i64
117func nx_cpcd_coupon_pos(c: *CpcDense, key: *u8, len: i64) -> i64
called by 1: nx_cpcd_add calls 2: murmur3_32nx_cpcd_clz32
128func nx_cpcd_add(c: *CpcDense, key: *u8, len: i64) -> i64
143func nx_cpcd_estimate(c: *CpcDense) -> i64
149func nx_cpcd_isqrt(x: i64) -> i64
168func nx_cpcd_stddev_rel_ppb(c: *CpcDense) -> i64
called by 1: nx_cpcd_query calls 1: nx_cpcd_isqrt
174func nx_cpcd_query(c: *CpcDense) -> *ApproxI64
187func nx_cpcd_merge(a: *CpcDense, b: *CpcDense) -> *CpcDense
222func nx_cpcd_n_set(c: *CpcDense) -> i64 { return c.n_set }
223func nx_cpcd_big_m(c: *CpcDense) -> i64 { return c.big_m }
226func nx_cpcd_memory_bytes(c: *CpcDense) -> i64
called by 2: mainmain