Fault Attribution on TRAIL (GAIA) 117 traces
63.25PrecisionSAFARI
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SAFARIModel=claude-opus-4.6, Context Budget=200K token2026.06 | 63.25 | 52.99 | |
| RAFFLESModel=claude-opus-4.6, Context Budget=1M token2026.06 | 61.54 | 56.41 | |
| SAFARIModel=claude-opus-4.6, Context Budget=100K token2026.06 | 61.54 | 50.43 | |
| SAFARIModel=claude-opus-4.6, Context Budget=1M token2026.06 | 58.97 | 51.28 | |
| RAFFLESModel=claude-opus-4.6, Context Budget=200K token2026.06 | 58.97 | 51.28 | |
| single shotModel=claude-opus-4.6, Context Budget=1M token2026.06 | 56.41 | 47.86 | |
| SAFARIModel=claude-opus-4.6, Context Budget=25K token2026.06 | 55.56 | 50.43 | |
| RAFFLESModel=claude-opus-4.6, Context Budget=100K token2026.06 | 54.7 | 48.72 | |
| single shotModel=claude-opus-4.6, Context Budget=200K token2026.06 | 52.14 | 44.44 | |
| RAFFLESModel=claude-opus-4.6, Context Budget=25K token2026.06 | 52.14 | 41.88 | |
| single shotModel=claude-opus-4.6, Context Budget=100K token2026.06 | 51.28 | 42.74 | |
| single shotModel=claude-opus-4.6, Context Budget=25K token2026.06 | 47.68 | 40.17 | |
| Step-by-stepModel=claude-opus-4.6, Context Budget=1M token2026.06 | 39.32 | 35.04 | |
| Step-by-stepModel=claude-opus-4.6, Context Budget=200K token2026.06 | 38.46 | 34.19 | |
| Step-by-stepModel=claude-opus-4.6, Context Budget=100K token2026.06 | 37.61 | 34.19 | |
| Step-by-stepModel=claude-opus-4.6, Context Budget=25K token2026.06 | 37.61 | 34.19 |