Failure Attribution Accuracy on Who&When (Alg)
42.86Accuracy (Who&When Alg)AgenTracer
Evaluation Results
| Method | Links | |
|---|---|---|
| AgenTracerevaluation with ground truth (G)=true2026.06 | 42.86 | |
| AgenTracer2026.06 | 37.3 | |
| CDC-MAS2026.06 | 36.2 | |
| StepFinder2026.06 | 29.63 | |
| StepFinderCategory=Sequential, Embedding=Qwen3-Embedding-0.6B, Temporal Feature Extractor=BiLSTM2026.06 | 29.63 | |
| TransformerCategory=Sequential, Layers=2, Attention Heads=42026.06 | 24.87 | |
| TCNCategory=Sequential, Layers=2, Kernel Size=32026.06 | 24.34 | |
| BiGRUCategory=Sequential, Layers=22026.06 | 23.28 | |
| Step-by-Step (G)Category=LLM-based, LLM Model=GPT-4o, Ground Truth=True2026.06 | 20.11 | |
| Binary Search (G)Category=LLM-based, LLM Model=GPT-4o, Ground Truth=True2026.06 | 20.1 | |
| All-at-Once (G)Category=LLM-based, LLM Model=GPT-4o, Ground Truth=True2026.06 | 15.87 | |
| Binary SearchCategory=LLM-based, LLM Model=GPT-4o, Ground Truth=False2026.06 | 15.61 | |
| RandomCategory=Baseline2026.06 | 15.08 | |
| Step-by-StepCategory=LLM-based, LLM Model=GPT-4o, Ground Truth=False2026.06 | 14.02 | |
| All-at-OnceCategory=LLM-based, LLM Model=GPT-4o, Ground Truth=False2026.06 | 11.9 |