# Failure Attribution on Who&When (HC)

Source: [Failure Attribution on Who&When (HC)](<https://www.sota2.com/research/sota/failure-attribution-on-who-and-when-hc>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-04T09:28:58.600Z

Evaluation of model performance on Failure Attribution using Accuracy (%) on the Who&When (HC) dataset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Failure Attribution |
| Dataset (including any stated split) | Who&When (HC) |
| Metric | Accuracy (%) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy (%)](<https://www.sota2.com/research/sota/failure-attribution-on-who-and-when-hc.md?metric=Accuracy%20(%25)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy (%) | Result date | Report paper |
| --- | --- | --- | --- |
| StepFinder | 22.99 | Not recorded | 6a2106a2c10484bac23f0291 |
| StepFinder | 22.99 | Not recorded | 6a2106a2c10484bac23f0291 |
| AgenTracer | 20.68 | Not recorded | 6a2106a2c10484bac23f0291 |
| AgenTracer | 20.68 | Not recorded | 6a2106a2c10484bac23f0291 |
| CDC-MAS | 18.2 | Not recorded | 6a2106a2c10484bac23f0291 |
| BiGRU | 12.64 | Not recorded | 6a2106a2c10484bac23f0291 |
| TCN | 10.35 | Not recorded | 6a2106a2c10484bac23f0291 |
| Transformer | 9.19 | Not recorded | 6a2106a2c10484bac23f0291 |
| Step-by-Step | 7.47 | Not recorded | 6a2106a2c10484bac23f0291 |
| Step-by-Step (G) | 6.9 | Not recorded | 6a2106a2c10484bac23f0291 |
| Binary Search | 6.32 | Not recorded | 6a2106a2c10484bac23f0291 |
| Binary Search (G) | 5.75 | Not recorded | 6a2106a2c10484bac23f0291 |
| Random | 4.6 | Not recorded | 6a2106a2c10484bac23f0291 |
| All-at-Once (G) | 4.6 | Not recorded | 6a2106a2c10484bac23f0291 |
| All-at-Once | 2.87 | Not recorded | 6a2106a2c10484bac23f0291 |

### StepFinder

Accuracy (%): 22.99. Result ID: 6a21453efbc373592c1d2efb.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### StepFinder

Accuracy (%): 22.99. Result ID: 6a21453efbc373592c1d2f11.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Sequential","Embedding":"Qwen3-Embedding-0.6B","Temporal Feature Extractor":"BiLSTM"}. This record does not assert independent reproduction.

### AgenTracer

Accuracy (%): 20.68. Result ID: 6a21453efbc373592c1d2ef8.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b251e209131cea86d99.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### AgenTracer

Accuracy (%): 20.68. Result ID: 6a21453efbc373592c1d2ef9.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b251e209131cea86d99.md>)

Recorded evaluation configuration: {"evaluation with ground truth (G)":"true"}. This record does not assert independent reproduction.

### CDC-MAS

Accuracy (%): 18.2. Result ID: 6a21453efbc373592c1d2efa.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b481e209131cea87835.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### BiGRU

Accuracy (%): 12.64. Result ID: 6a21453efbc373592c1d2f0e.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Sequential","Layers":"2"}. This record does not assert independent reproduction.

### TCN

Accuracy (%): 10.35. Result ID: 6a21453efbc373592c1d2f0f.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Sequential","Layers":"2","Kernel Size":"3"}. This record does not assert independent reproduction.

### Transformer

Accuracy (%): 9.19. Result ID: 6a21453efbc373592c1d2f10.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Sequential","Layers":"2","Attention Heads":"4"}. This record does not assert independent reproduction.

### Step-by-Step

Accuracy (%): 7.47. Result ID: 6a21453efbc373592c1d2f09.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"False"}. This record does not assert independent reproduction.

### Step-by-Step (G)

Accuracy (%): 6.9. Result ID: 6a21453efbc373592c1d2f0c.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"True"}. This record does not assert independent reproduction.

### Binary Search

Accuracy (%): 6.32. Result ID: 6a21453efbc373592c1d2f0a.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"False"}. This record does not assert independent reproduction.

### Binary Search (G)

Accuracy (%): 5.75. Result ID: 6a21453efbc373592c1d2f0d.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"True"}. This record does not assert independent reproduction.

### Random

Accuracy (%): 4.6. Result ID: 6a21453efbc373592c1d2f07.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Baseline"}. This record does not assert independent reproduction.

### All-at-Once (G)

Accuracy (%): 4.6. Result ID: 6a21453efbc373592c1d2f0b.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"True"}. This record does not assert independent reproduction.

### All-at-Once

Accuracy (%): 2.87. Result ID: 6a21453efbc373592c1d2f08.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"False"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
