Fault attribution on Who&When Hand (58 traces)
31Top AccuracyRAFFLES
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RAFFLESModel=claude-opus-4.6, Context size=200K token2026.06 | 31 | 39.7 | 39.7 | 41.4 | 48.3 | 50 | |
| RAFFLESModel=claude-opus-4.6, Context size=25K token2026.06 | 31 | 39.7 | 39.7 | 41.4 | 50 | 51.7 | |
| RAFFLESModel=claude-opus-4.6, Context size=100K token2026.06 | 29.3 | 39.7 | 39.7 | 41.4 | 48.3 | 48.3 | |
| SAFARIModel=claude-opus-4.6, Context size=100K token2026.06 | 27.6 | 41.4 | 41.4 | 48.3 | 55.2 | 55.2 | |
| RAFFLESModel=claude-opus-4.6, Context size=1M token2026.06 | 25.9 | 34.5 | 34.5 | 37.9 | 46.6 | 46.6 | |
| SAFARIModel=claude-opus-4.6, Context size=1M token2026.06 | 25.9 | 41.4 | 41.4 | 46.6 | 56.9 | 56.9 | |
| SAFARIModel=claude-opus-4.6, Context size=25K token2026.06 | 25.9 | 36.2 | 37.9 | 43.1 | 53.4 | 53.4 | |
| SAFARIModel=claude-opus-4.6, Context size=200K token2026.06 | 22.4 | 37.9 | 37.9 | 43.1 | 55.2 | 55.2 | |
| single shotModel=claude-opus-4.6, Context size=25K token2026.06 | 20.7 | 29.3 | 29.3 | 39.7 | 53.4 | 56.9 | |
| single shotModel=claude-opus-4.6, Context size=1M token2026.06 | 15.5 | 24.1 | 24.1 | 36.2 | 48.3 | 50 | |
| single shotModel=claude-opus-4.6, Context size=200K token2026.06 | 15.5 | 24.1 | 24.1 | 34.5 | 50 | 51.7 | |
| single shotModel=claude-opus-4.6, Context size=100K token2026.06 | 15.5 | 20.7 | 20.7 | 32.8 | 46.6 | 50 | |
| Step-by-stepModel=claude-opus-4.6, Context size=100K token2026.06 | 13.8 | 20.7 | 25.9 | 31 | 46.6 | 50 | |
| Step-by-stepModel=claude-opus-4.6, Context size=25K token2026.06 | 13.8 | 20.7 | 25.9 | 31 | 46.6 | 50 | |
| Step-by-stepModel=claude-opus-4.6, Context size=1M token2026.06 | 12.1 | 19 | 24.1 | 31 | 46.6 | 50 | |
| Step-by-stepModel=claude-opus-4.6, Context size=200K token2026.06 | 12.1 | 20.7 | 25.9 | 32.8 | 48.3 | 51.7 |