nx_verdictlog_lib.nx

nx_verdictlog_lib.nx -- THE VERDICT-LOG RULER: read a detector's own recorded verdicts and say what its output is worth. FM3 and FM4 of /compare/failmodes, and ONE ruler for both because both ask the same primitive question of the same artifacts: WHAT VERDICT DID THIS DETECTOR EMIT, AND WHEN. FM3 reads one stream through time (is this detector's output informative). FM4 reads two streams of the same subject side by side (do two vantages agree). Splitting the parser between them would be the duplicate-ruler defect, and the census and its gate would then be free to disagree about what a verdict IS. --------------------------------------------------------------------------------------------- FM3 -- THE ACCEPT RULE, QUOTED VERBATIM FROM failmodes.plan BEFORE THE WORK BEGAN: "run it over the EXISTING detector fleet and publish the four numbers per detector; the rung is accepted only if at least one currently-shipping detector is shown to score badly enough to justify changing or retiring it. A scorer that flatters every incumbent has measured nothing." THE FOUR NUMBERS ARE SRE'S -- precision, recall, detection time, reset time -- and EACH IS THREE-STATE. That is not a softening of the rule, it is the only honest way to publish it, and the reason is measured rather than assumed: A VERDICT LOG RECORDS WHAT THE DETECTOR SAID AND NEVER WHETHER IT WAS RIGHT. Nothing in this estate joins a detector's firing to an adjudication, so a precision figure computed from these files would be a CONSTANT WEARING THE SHAPE OF A MEASUREMENT. Each axis therefore carries either a value or a NAMED abstention: precision UNINFORMATIVE-CONSTANT the output never varies, so it carries no information and its precision equals the base rate. This is an EXACT statement, not an estimate. UNOBSERVABLE-NO-ADJUDICATION otherwise: nothing joins a firing to a truth. recall UNVERIFIED-NEVER-FIRED the detector has produced no positive in its whole recorded history, so nothing has demonstrated it CAN fire. A green that never had a corresponding red is unverified. UNOBSERVABLE-NO-GROUND-TRUTH otherwise: a missed event leaves no trace here. detect_bound_s <n> the largest observed interval between consecutive runs. A BOUND, and labelled one: a condition that becomes visible just after a run waits at most this long to be seen. UNOBSERVABLE-NO-TIMESTAMP the stream carries no declared timestamp key. reset <n> the largest observed number of runs from a non-green to the next green. LATCHED the trailing non-green episode never returns to