code wiki / _hdl_build / nx_curate.nx

nx_curate.nx

buildroot/runtime/_hdl_build/nx_curate.nx

6395 B110 linesdepth 4pulls 4 transitivereach 0 importersview sourcekind tool
docsdependenciesstructsconstsfunctions

about

nx_curate.nx -- MLIB-005 (CALLOUT-012). The CURATION / DEDUP / QUALITY-JUDGE engine ("detectors in reverse", COMP-003 pattern): mechanical keep/remove decisions over a media corpus. Powers find-fast + remove-fast. Composes the PROVEN nx_referee_v2 (rf2_tpr/tnr/ testset_trustworthy) for precision/recall -- the engine does NOT self-grade. - EXACT dedup: same content fingerprint (MLIB-003 nx_media_record CID) -> duplicate. - NEAR dedup: perceptual-hash Hamming distance <= NEAR_THRESH -> near-duplicate. - QUALITY: keep-score < QUAL_THRESH -> low-quality remove candidate (thresholds = config, rule 11; here as named consts, store-backed later). - CORRUPTION: corrupt flag -> remove candidate. REMOVE decisions are emitted as SOFT-DELETE PROPOSALS (is_current=0, rule 13 -- ADDITIVE, NEVER hard delete; the operator/store keeps history). Self-validating gate (CURATEGATE): precision/recall == 1000permil on a seeded ground-truth corpus + near/far/exact controls. license_tier: ORIGINAL

dependencies 2 imports · 0 importers

nx_syscalls.nx nx_referee_v2.nx nx_curate.nx

imports: nx_syscalls.nxnx_referee_v2.nx

imported by: nobody (leaf or entry point)

call flow from main pre-order; caps 40 nodes / depth 6 declared; ↻ = already shown

main sys_mmap cu_streq cu_isneardup cu_hamming cu_pop4 cu_hexval cu_puts sys_write cu_putn sys_mmap ↻ sys_write ↻ rf2_tpr rf2_permil rf2_tnr rf2_permil ↻ rf2_testset_trustworthy

structs

none

consts

17const CU_N: i64 = 7
18const NEAR_THRESH: i64 = 4 // <=4 differing bits in the phash = near-duplicate
19const QUAL_THRESH: i64 = 500 // keep-score below this = low-quality remove candidate

functions

21func cu_puts(s: *u8) -> i64 { var n: i64 = 0; while s[n] != (0 as u8) { n = n + 1 } sys_write(1, s, n); return 0 }
called by 1: main calls 1: sys_write
22func cu_putn(v: i64) -> i64 { let bb: *u8 = sys_mmap(28); var m: i64=v; if m<0 {m=0-m}; let t: *u8 = sys_mmap(28); var k: i64=0; if m==0 {t[0]=48;k=1}; while m>0 {t[k]=48+(m%10); m=m/10; k=k+1}; var i: i64=0; while i<k {bb[i]=t[k-1-i]; i=i+1}; sys_write(1, bb, k); return 0 }
called by 1: main calls 2: sys_mmapsys_write
24func cu_streq(a: *u8, b: *u8) -> i64 { var i: i64 = 0; var s: i64 = 1; while s == 1 { if a[i] != b[i] { s = 0 } else { if a[i] == (0 as u8) { return 1 } else { i = i + 1 } } } return 0 }
called by 1: main
25func cu_hexval(c: i64) -> i64 { if c >= 97 { return (c - 97) + 10 } if c >= 65 { return (c - 65) + 10 } return c - 48 }
called by 1: cu_hamming
26func cu_pop4(x: i64) -> i64 { var c: i64 = 0; var v: i64 = x; var i: i64 = 0; while i < 4 { c = c + (v & 1); v = v >> 1; i = i + 1 } return c }
called by 1: cu_hamming
29func cu_hamming(a: *u8, b: *u8) -> i64
called by 1: cu_isneardup calls 2: cu_pop4cu_hexval
36func cu_isneardup(a: *u8, b: *u8) -> i64 { if cu_hamming(a, b) <= NEAR_THRESH { return 1 } return 0 }
called by 1: main calls 1: cu_hamming
38func main() -> i64