# Fault attribution on Who&When Algo (126 traces) Source: [Fault attribution on Who&When Algo (126 traces)]() Collection index: [llms.txt]() Data updated: 2026-06-24T08:42:52.237Z Evaluation of Fault Attribution performance on the specific 'Who&When Algo (126 traces)' dataset, reporting overall accuracy and tolerance-based accuracies (±1 through ±5). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Fault attribution | | Dataset (including any stated split) | Who&When Algo (126 traces) | | Metric | Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Accuracy](): Higher is better; unit: PERCENT. - [Accuracy (±1 Tolerance)](): Higher is better; unit: PERCENT. - [Accuracy (±2 Tolerance)](): Higher is better; unit: PERCENT. - [Accuracy (±3 Tolerance)](): Higher is better; unit: PERCENT. - [Accuracy (±4 Tolerance)](): Higher is better; unit: PERCENT. - [Accuracy (±5 Tolerance)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Accuracy | Result date | Report paper | | --- | --- | --- | --- | | RAFFLES | 49.2 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 49.2 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 48.4 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 48.4 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 48.4 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 47.6 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 46.8 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 46 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 46 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 45.2 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 45.2 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 44.4 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 34.9 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 34.1 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 33.3 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 33.3 | Not recorded | 6a3b64dd206542251d7c999f | ### RAFFLES Accuracy: 49.2. Result ID: 6a3b9880fbc373592c44d73a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"100K token"}. This record does not assert independent reproduction. ### RAFFLES Accuracy: 49.2. Result ID: 6a3b9880fbc373592c44d73e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"25K token"}. This record does not assert independent reproduction. ### SAFARI Accuracy: 48.4. Result ID: 6a3b9880fbc373592c44d733. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"1M token"}. This record does not assert independent reproduction. ### SAFARI Accuracy: 48.4. Result ID: 6a3b9880fbc373592c44d737. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"200K token"}. This record does not assert independent reproduction. ### SAFARI Accuracy: 48.4. Result ID: 6a3b9880fbc373592c44d73f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"25K token"}. This record does not assert independent reproduction. ### SAFARI Accuracy: 47.6. Result ID: 6a3b9880fbc373592c44d73b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"100K token"}. This record does not assert independent reproduction. ### single shot Accuracy: 46.8. Result ID: 6a3b9880fbc373592c44d73c. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"25K token"}. This record does not assert independent reproduction. ### single shot Accuracy: 46. Result ID: 6a3b9880fbc373592c44d734. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"200K token"}. This record does not assert independent reproduction. ### single shot Accuracy: 46. Result ID: 6a3b9880fbc373592c44d738. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"100K token"}. This record does not assert independent reproduction. ### RAFFLES Accuracy: 45.2. Result ID: 6a3b9880fbc373592c44d732. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"1M token"}. This record does not assert independent reproduction. ### RAFFLES Accuracy: 45.2. Result ID: 6a3b9880fbc373592c44d736. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"200K token"}. This record does not assert independent reproduction. ### single shot Accuracy: 44.4. Result ID: 6a3b9880fbc373592c44d730. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"1M token"}. This record does not assert independent reproduction. ### Step-by-step Accuracy: 34.9. Result ID: 6a3b9880fbc373592c44d731. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"1M token"}. This record does not assert independent reproduction. ### Step-by-step Accuracy: 34.1. Result ID: 6a3b9880fbc373592c44d735. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"200K token"}. This record does not assert independent reproduction. ### Step-by-step Accuracy: 33.3. Result ID: 6a3b9880fbc373592c44d739. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"100K token"}. This record does not assert independent reproduction. ### Step-by-step Accuracy: 33.3. Result ID: 6a3b9880fbc373592c44d73d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"25K token"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.