# Fault Attribution on TRAIL (GAIA) 117 traces Source: [Fault Attribution on TRAIL (GAIA) 117 traces]() Collection index: [llms.txt]() Data updated: 2026-06-24T08:42:52.304Z Evaluation of model performance on the Fault Attribution task using the specific 117 traces subset of the TRAIL GAIA dataset, measured by Precision and Strict Precision. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Fault Attribution | | Dataset (including any stated split) | TRAIL (GAIA) 117 traces | | Metric | Precision | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Precision](): Higher is better; unit: PERCENT. - [Strict Precision](): Higher is better; unit: PERCENT. ## Recorded results | Method | Precision | Result date | Report paper | | --- | --- | --- | --- | | SAFARI | 63.25 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 61.54 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 61.54 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 58.97 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 58.97 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 56.41 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 55.56 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 54.7 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 52.14 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 52.14 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 51.28 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 47.68 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 39.32 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 38.46 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 37.61 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 37.61 | Not recorded | 6a3b64dd206542251d7c999f | ### SAFARI Precision: 63.25. Result ID: 6a3b9880fbc373592c44d757. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"200K token"}. This record does not assert independent reproduction. ### RAFFLES Precision: 61.54. Result ID: 6a3b9880fbc373592c44d752. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"1M token"}. This record does not assert independent reproduction. ### SAFARI Precision: 61.54. Result ID: 6a3b9880fbc373592c44d75b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"100K token"}. This record does not assert independent reproduction. ### SAFARI Precision: 58.97. Result ID: 6a3b9880fbc373592c44d753. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"1M token"}. This record does not assert independent reproduction. ### RAFFLES Precision: 58.97. Result ID: 6a3b9880fbc373592c44d756. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"200K token"}. This record does not assert independent reproduction. ### single shot Precision: 56.41. Result ID: 6a3b9880fbc373592c44d750. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"1M token"}. This record does not assert independent reproduction. ### SAFARI Precision: 55.56. Result ID: 6a3b9880fbc373592c44d75f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"25K token"}. This record does not assert independent reproduction. ### RAFFLES Precision: 54.7. Result ID: 6a3b9880fbc373592c44d75a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"100K token"}. This record does not assert independent reproduction. ### single shot Precision: 52.14. Result ID: 6a3b9880fbc373592c44d754. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"200K token"}. This record does not assert independent reproduction. ### RAFFLES Precision: 52.14. Result ID: 6a3b9880fbc373592c44d75e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"25K token"}. This record does not assert independent reproduction. ### single shot Precision: 51.28. Result ID: 6a3b9880fbc373592c44d758. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"100K token"}. This record does not assert independent reproduction. ### single shot Precision: 47.68. Result ID: 6a3b9880fbc373592c44d75c. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"25K token"}. This record does not assert independent reproduction. ### Step-by-step Precision: 39.32. Result ID: 6a3b9880fbc373592c44d751. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"1M token"}. This record does not assert independent reproduction. ### Step-by-step Precision: 38.46. Result ID: 6a3b9880fbc373592c44d755. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"200K token"}. This record does not assert independent reproduction. ### Step-by-step Precision: 37.61. Result ID: 6a3b9880fbc373592c44d759. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"100K token"}. This record does not assert independent reproduction. ### Step-by-step Precision: 37.61. Result ID: 6a3b9880fbc373592c44d75d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context Budget":"25K token"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.