# Failure Attribution Accuracy on Who&When (Alg)

Source: [Failure Attribution Accuracy on Who&When (Alg)](<https://www.sota2.com/research/sota/failure-attribution-accuracy-on-who-and-when-alg>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-04T09:28:58.573Z

Evaluation of model performance using Accuracy (%) on the Failure Attribution task applied to the Who&When (Alg) dataset subset.

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | Failure Attribution |
| Dataset (including any stated split) | Who&When (Alg) |
| Metric | Accuracy (Who&When Alg) |
| Unit | PERCENT |
| Direction | Higher is better |
| Results loaded | 15 of 15 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Accuracy (Who&When Alg)](<https://www.sota2.com/research/sota/failure-attribution-accuracy-on-who-and-when-alg.md?metric=Accuracy%20(Who%26When%20Alg)>): Higher is better; unit: PERCENT.

## Recorded results

| Method | Accuracy (Who&When Alg) | Result date | Report paper |
| --- | --- | --- | --- |
| AgenTracer | 42.86 | Not recorded | 6a2106a2c10484bac23f0291 |
| AgenTracer | 37.3 | Not recorded | 6a2106a2c10484bac23f0291 |
| CDC-MAS | 36.2 | Not recorded | 6a2106a2c10484bac23f0291 |
| StepFinder | 29.63 | Not recorded | 6a2106a2c10484bac23f0291 |
| StepFinder | 29.63 | Not recorded | 6a2106a2c10484bac23f0291 |
| Transformer | 24.87 | Not recorded | 6a2106a2c10484bac23f0291 |
| TCN | 24.34 | Not recorded | 6a2106a2c10484bac23f0291 |
| BiGRU | 23.28 | Not recorded | 6a2106a2c10484bac23f0291 |
| Step-by-Step (G) | 20.11 | Not recorded | 6a2106a2c10484bac23f0291 |
| Binary Search (G) | 20.1 | Not recorded | 6a2106a2c10484bac23f0291 |
| All-at-Once (G) | 15.87 | Not recorded | 6a2106a2c10484bac23f0291 |
| Binary Search | 15.61 | Not recorded | 6a2106a2c10484bac23f0291 |
| Random | 15.08 | Not recorded | 6a2106a2c10484bac23f0291 |
| Step-by-Step | 14.02 | Not recorded | 6a2106a2c10484bac23f0291 |
| All-at-Once | 11.9 | Not recorded | 6a2106a2c10484bac23f0291 |

### AgenTracer

Accuracy (Who&When Alg): 42.86. Result ID: 6a21453efbc373592c1d2ef5.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b251e209131cea86d99.md>)

Recorded evaluation configuration: {"evaluation with ground truth (G)":"true"}. This record does not assert independent reproduction.

### AgenTracer

Accuracy (Who&When Alg): 37.3. Result ID: 6a21453efbc373592c1d2ef4.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b251e209131cea86d99.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### CDC-MAS

Accuracy (Who&When Alg): 36.2. Result ID: 6a21453efbc373592c1d2ef6.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa7b481e209131cea87835.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### StepFinder

Accuracy (Who&When Alg): 29.63. Result ID: 6a21453efbc373592c1d2ef7.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction.

### StepFinder

Accuracy (Who&When Alg): 29.63. Result ID: 6a21453efbc373592c1d2f06.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Sequential","Embedding":"Qwen3-Embedding-0.6B","Temporal Feature Extractor":"BiLSTM"}. This record does not assert independent reproduction.

### Transformer

Accuracy (Who&When Alg): 24.87. Result ID: 6a21453efbc373592c1d2f05.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Sequential","Layers":"2","Attention Heads":"4"}. This record does not assert independent reproduction.

### TCN

Accuracy (Who&When Alg): 24.34. Result ID: 6a21453efbc373592c1d2f04.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Sequential","Layers":"2","Kernel Size":"3"}. This record does not assert independent reproduction.

### BiGRU

Accuracy (Who&When Alg): 23.28. Result ID: 6a21453efbc373592c1d2f03.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Sequential","Layers":"2"}. This record does not assert independent reproduction.

### Step-by-Step (G)

Accuracy (Who&When Alg): 20.11. Result ID: 6a21453efbc373592c1d2f01.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"True"}. This record does not assert independent reproduction.

### Binary Search (G)

Accuracy (Who&When Alg): 20.1. Result ID: 6a21453efbc373592c1d2f02.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"True"}. This record does not assert independent reproduction.

### All-at-Once (G)

Accuracy (Who&When Alg): 15.87. Result ID: 6a21453efbc373592c1d2f00.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"True"}. This record does not assert independent reproduction.

### Binary Search

Accuracy (Who&When Alg): 15.61. Result ID: 6a21453efbc373592c1d2eff.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"False"}. This record does not assert independent reproduction.

### Random

Accuracy (Who&When Alg): 15.08. Result ID: 6a21453efbc373592c1d2efc.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

Recorded evaluation configuration: {"Category":"Baseline"}. This record does not assert independent reproduction.

### Step-by-Step

Accuracy (Who&When Alg): 14.02. Result ID: 6a21453efbc373592c1d2efe.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"False"}. This record does not assert independent reproduction.

### All-at-Once

Accuracy (Who&When Alg): 11.9. Result ID: 6a21453efbc373592c1d2efd.

[Reporting paper](<https://www.sota2.com/research/paper/6a2106a2c10484bac23f0291.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa790b1e209131cea7bda8.md>)

Recorded evaluation configuration: {"Category":"LLM-based","LLM Model":"GPT-4o","Ground Truth":"False"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
