Failure Attribution on Who&When (HC)
22.99Accuracy (%)StepFinder
Evaluation Results
| Method | Links | |
|---|---|---|
| StepFinder2026.06 | 22.99 | |
| StepFinderCategory=Sequential, Embedding=Qwen3-Embedding-0.6B, Temporal Feature Extractor=BiLSTM2026.06 | 22.99 | |
| AgenTracer2026.06 | 20.68 | |
| AgenTracerevaluation with ground truth (G)=true2026.06 | 20.68 | |
| CDC-MAS2026.06 | 18.2 | |
| BiGRUCategory=Sequential, Layers=22026.06 | 12.64 | |
| TCNCategory=Sequential, Layers=2, Kernel Size=32026.06 | 10.35 | |
| TransformerCategory=Sequential, Layers=2, Attention Heads=42026.06 | 9.19 | |
| Step-by-StepCategory=LLM-based, LLM Model=GPT-4o, Ground Truth=False2026.06 | 7.47 | |
| Step-by-Step (G)Category=LLM-based, LLM Model=GPT-4o, Ground Truth=True2026.06 | 6.9 | |
| Binary SearchCategory=LLM-based, LLM Model=GPT-4o, Ground Truth=False2026.06 | 6.32 | |
| Binary Search (G)Category=LLM-based, LLM Model=GPT-4o, Ground Truth=True2026.06 | 5.75 | |
| RandomCategory=Baseline2026.06 | 4.6 | |
| All-at-Once (G)Category=LLM-based, LLM Model=GPT-4o, Ground Truth=True2026.06 | 4.6 | |
| All-at-OnceCategory=LLM-based, LLM Model=GPT-4o, Ground Truth=False2026.06 | 2.87 |