Error Localization on BBM
60.1EMAAO (Opus 4.6)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AAO (Opus 4.6)Auditor model=Claude Opus 4.62026.06 | 60.1 | 70.6 | |
| REFLECTAuditor model=gpt-5.2, Verification source=Oracle2026.06 | 34.5 | 51.8 | |
| Reflexion+GTAuditor model=gpt-5.2, Ground Truth availability=Yes2026.06 | 34 | 52.7 | |
| REFLECT (proxy)Auditor model=gpt-5.2, Verification source=Proxy2026.06 | 31 | 45.1 | |
| AAOAuditor model=gpt-5.22026.06 | 30.3 | 47.2 | |
| ICSAuditor model=gpt-5.22026.06 | 19.7 | 31.7 | |
| BSAuditor model=gpt-5.22026.06 | 16.2 | 29.6 | |
| ThinkPRMAuditor model=gpt-5.22026.06 | 14.7 | 34.7 | |
| Reflexion–noGTAuditor model=gpt-5.2, Ground Truth availability=No2026.06 | 14 | 33.3 | |
| AgentRxAuditor model=gpt-5.22026.06 | 3.3 | 29.3 | |
| DoVer+GTAuditor model=gpt-5.2, Ground Truth availability=Yes2026.06 | 2 | 25.3 | |
| SBSAuditor model=gpt-5.22026.06 | 0.7 | 26.1 |