# Fault attribution on Who&When Hand (58 traces) Source: [Fault attribution on Who&When Hand (58 traces)]() Collection index: [llms.txt]() Data updated: 2026-06-24T08:42:52.275Z Evaluation of fault attribution performance using Top-K accuracy metrics (up to K=5) on the Who&When Hand (58 traces) dataset. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Fault attribution | | Dataset (including any stated split) | Who&When Hand (58 traces) | | Metric | Top Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Top Accuracy](): Higher is better; unit: PERCENT. - [Top-1 Accuracy](): Higher is better; unit: PERCENT. - [Top-2 Accuracy](): Higher is better; unit: PERCENT. - [Top-3 Accuracy](): Higher is better; unit: PERCENT. - [Top-4 Accuracy](): Higher is better; unit: PERCENT. - [Top-5 Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | Top Accuracy | Result date | Report paper | | --- | --- | --- | --- | | RAFFLES | 31 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 31 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 29.3 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 27.6 | Not recorded | 6a3b64dd206542251d7c999f | | RAFFLES | 25.9 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 25.9 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 25.9 | Not recorded | 6a3b64dd206542251d7c999f | | SAFARI | 22.4 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 20.7 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 15.5 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 15.5 | Not recorded | 6a3b64dd206542251d7c999f | | single shot | 15.5 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 13.8 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 13.8 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 12.1 | Not recorded | 6a3b64dd206542251d7c999f | | Step-by-step | 12.1 | Not recorded | 6a3b64dd206542251d7c999f | ### RAFFLES Top Accuracy: 31. Result ID: 6a3b9880fbc373592c44d746. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"200K token"}. This record does not assert independent reproduction. ### RAFFLES Top Accuracy: 31. Result ID: 6a3b9880fbc373592c44d74e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"25K token"}. This record does not assert independent reproduction. ### RAFFLES Top Accuracy: 29.3. Result ID: 6a3b9880fbc373592c44d74a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"100K token"}. This record does not assert independent reproduction. ### SAFARI Top Accuracy: 27.6. Result ID: 6a3b9880fbc373592c44d74b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"100K token"}. This record does not assert independent reproduction. ### RAFFLES Top Accuracy: 25.9. Result ID: 6a3b9880fbc373592c44d742. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"1M token"}. This record does not assert independent reproduction. ### SAFARI Top Accuracy: 25.9. Result ID: 6a3b9880fbc373592c44d743. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"1M token"}. This record does not assert independent reproduction. ### SAFARI Top Accuracy: 25.9. Result ID: 6a3b9880fbc373592c44d74f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"25K token"}. This record does not assert independent reproduction. ### SAFARI Top Accuracy: 22.4. Result ID: 6a3b9880fbc373592c44d747. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"200K token"}. This record does not assert independent reproduction. ### single shot Top Accuracy: 20.7. Result ID: 6a3b9880fbc373592c44d74c. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"25K token"}. This record does not assert independent reproduction. ### single shot Top Accuracy: 15.5. Result ID: 6a3b9880fbc373592c44d740. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"1M token"}. This record does not assert independent reproduction. ### single shot Top Accuracy: 15.5. Result ID: 6a3b9880fbc373592c44d744. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"200K token"}. This record does not assert independent reproduction. ### single shot Top Accuracy: 15.5. Result ID: 6a3b9880fbc373592c44d748. [Reporting paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"100K token"}. This record does not assert independent reproduction. ### Step-by-step Top Accuracy: 13.8. Result ID: 6a3b9880fbc373592c44d749. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"100K token"}. This record does not assert independent reproduction. ### Step-by-step Top Accuracy: 13.8. Result ID: 6a3b9880fbc373592c44d74d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"25K token"}. This record does not assert independent reproduction. ### Step-by-step Top Accuracy: 12.1. Result ID: 6a3b9880fbc373592c44d741. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"1M token"}. This record does not assert independent reproduction. ### Step-by-step Top Accuracy: 12.1. Result ID: 6a3b9880fbc373592c44d745. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"claude-opus-4.6","Context size":"200K token"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.