Failure Attribution on Who&When
60.79Agent AccuracyAEA-4B
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AEA-4BReasoning Strategy=All at Once2026.05 | 60.79 | 32.7 | |
| Claude 3.7 SonnetReasoning Strategy=All at Once2026.05 | 58.7 | 24.6 | |
| Qwen2.5-72BReasoning Strategy=All at Once2026.05 | 55.56 | 10.32 | |
| Qwen3-32BReasoning Strategy=All at Once2026.05 | 53.97 | 15.87 | |
| GPT-4oReasoning Strategy=All at Once2026.05 | 50 | 30.95 | |
| Llama-3.1-70BReasoning Strategy=All at Once2026.05 | 48.41 | 16.67 | |
| AEA-4BReasoning Strategy=Step by Step2026.05 | 48.41 | 25.87 | |
| Claude 3.7 SonnetReasoning Strategy=Step by Step2026.05 | 46.03 | 30.95 | |
| Qwen3-32BReasoning Strategy=Step by Step2026.05 | 38.1 | 23.02 | |
| GPT-4o-miniReasoning Strategy=All at Once2026.05 | 35.71 | 19.84 | |
| GPT-4oReasoning Strategy=Step by Step2026.05 | 33.33 | 26.43 | |
| Qwen3-8BReasoning Strategy=All at Once2026.05 | 31.75 | 5.56 | |
| Llama-3.1-70BReasoning Strategy=Step by Step2026.05 | 30.95 | 19.84 | |
| GPT-4o-miniReasoning Strategy=Step by Step2026.05 | 30.95 | 19.05 | |
| Llama-3.1-8BReasoning Strategy=Step by Step2026.05 | 24.6 | 13.49 | |
| Claude 3.5 HaikuReasoning Strategy=Step by Step2026.05 | 21.43 | 16.67 | |
| Qwen2.5-72BReasoning Strategy=Step by Step2026.05 | 20.63 | 16.67 | |
| Qwen3-8BReasoning Strategy=Step by Step2026.05 | 20.63 | 10.32 | |
| Claude 3.5 HaikuReasoning Strategy=All at Once2026.05 | 12.7 | 7.94 | |
| Qwen2.5-7BReasoning Strategy=Step by Step2026.05 | 11.11 | 6.35 | |
| Qwen2.5-7BReasoning Strategy=All at Once2026.05 | 10.32 | 5.56 | |
| Llama-3.1-8BReasoning Strategy=All at Once2026.05 | 9.52 | 2.38 |