Fault attribution on Who&When Algo (126 traces)
49.2AccuracyRAFFLES
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| RAFFLESModel=claude-opus-4.6, Context size=100K token2026.06 | 49.2 | 55.6 | 71.4 | 78.6 | 81.8 | 87.3 | |
| RAFFLESModel=claude-opus-4.6, Context size=25K token2026.06 | 49.2 | 55.6 | 71.4 | 78.6 | 81.8 | 87.3 | |
| SAFARIModel=claude-opus-4.6, Context size=1M token2026.06 | 48.4 | 57.9 | 73 | 80.9 | 85.7 | 92.1 | |
| SAFARIModel=claude-opus-4.6, Context size=200K token2026.06 | 48.4 | 59.5 | 74.6 | 81.8 | 85.7 | 92.1 | |
| SAFARIModel=claude-opus-4.6, Context size=25K token2026.06 | 48.4 | 57.1 | 72.2 | 79.4 | 84.1 | 90.5 | |
| SAFARIModel=claude-opus-4.6, Context size=100K token2026.06 | 47.6 | 56.4 | 73 | 80.9 | 85.7 | 91.3 | |
| single shotModel=claude-opus-4.6, Context size=25K token2026.06 | 46.8 | 54.8 | 68.2 | 76.2 | 81.8 | 88.9 | |
| single shotModel=claude-opus-4.6, Context size=200K token2026.06 | 46 | 54 | 68.2 | 76.2 | 81.8 | 89.7 | |
| single shotModel=claude-opus-4.6, Context size=100K token2026.06 | 46 | 54.8 | 68.3 | 76.3 | 81.8 | 89.7 | |
| RAFFLESModel=claude-opus-4.6, Context size=1M token2026.06 | 45.2 | 54.8 | 70.6 | 77.8 | 80.9 | 85.7 | |
| RAFFLESModel=claude-opus-4.6, Context size=200K token2026.06 | 45.2 | 54.8 | 69.1 | 76.2 | 80.9 | 84.9 | |
| single shotModel=claude-opus-4.6, Context size=1M token2026.06 | 44.4 | 52.4 | 67.5 | 75.4 | 80.9 | 88.9 | |
| Step-by-stepModel=claude-opus-4.6, Context size=1M token2026.06 | 34.9 | 52.4 | 62.7 | 69.8 | 77.8 | 88.1 | |
| Step-by-stepModel=claude-opus-4.6, Context size=200K token2026.06 | 34.1 | 50.8 | 61.9 | 69.8 | 77.8 | 88.1 | |
| Step-by-stepModel=claude-opus-4.6, Context size=100K token2026.06 | 33.3 | 52.4 | 63.9 | 71.4 | 78.6 | 88.1 | |
| Step-by-stepModel=claude-opus-4.6, Context size=25K token2026.06 | 33.3 | 52.4 | 63.5 | 71.4 | 78.6 | 88.1 |