Failure classification on LongMemEval failures 8B
76.2AccuracyCircuit diagnostic
Evaluation results
| Method | Links | |
|---|---|---|
| Circuit diagnosticDescription=Max-damage ablation of operation-specific features2026.05 | 76.2 | |
| Black-box classifierDescription=Logistic regression on pipeline features (5-fold CV)2026.05 | 63.4 | |
| Stagewise confidenceDescription=Predict the stage with highest output entropy2026.05 | 51.5 | |
| Majority classDescription=Always predict the most common failure stage2026.05 | 51 | |
| Behavioral ruleDescription=Decision tree over answer text heuristics2026.05 | 45.4 |