Classifier accuracy — public disclosure

Ship-gate thresholds

No new classifier prompt ships unless every metric passes on both eval sets.

• accuracy ≥ 0.95 — exact trade match
• macro F1 ≥ 0.90 — unweighted across 6 trade classes
• per-class recall ≥ 0.85 — every present class, prevents sparse-class collapse
• Brier score ≤ 0.05 — calibration: confidence vs is_correct
• ECE ≤ 0.10 — 10-bin expected calibration error

Latest results — loading…

Independence

Ground-truth eval sets are locked and append-only. No prompt change is allowed to mutate prior versions. CI gates on the highest version. Multi-rater protocol applies v2+: two independent LLM judges (Anthropic + OpenAI), human button-push only on disagreement. Full protocol.