Failure Attribution on Who&When Total
36.22Step-level AccuracyAgenTracer (G)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AgenTracer (G)Evaluation Mode=Offline, Method Paradigm=Finetune-Based, Ground-truth Guidance=true2026.03 | 36.22 | 69.73 | |
| AgenTracerEvaluation Mode=Offline, Method Paradigm=Finetune-Based, Ground-truth Guidance=false2026.03 | 31.89 | 63.78 | |
| FamasEvaluation Mode=Offline, Method Paradigm=SBFL-Based2026.03 | 29.35 | 57.61 | |
| PROMASEvaluation Mode=Online, Backbone=Meta-Llama-3.1-8B-Instruct2026.03 | 22.97 | 40.54 | |
| MASCEvaluation Mode=Online, Backbone=Meta-Llama-3.1-8B-Instruct, Ground-truth Guidance=false2026.03 | 21.62 | — | |
| Step-by-Step (G)Evaluation Mode=Offline, Method Paradigm=LLM-Based, Backbone=GPT-4o, Ground-truth Guidance=true2026.03 | 19.68 | 34.97 | |
| MASC (G)Evaluation Mode=Online, Backbone=Meta-Llama-3.1-8B-Instruct, Ground-truth Guidance=true2026.03 | 18.92 | — | |
| Binary Search (G)Evaluation Mode=Offline, Method Paradigm=LLM-Based, Backbone=GPT-4o, Ground-truth Guidance=true2026.03 | 18.6 | 46.52 | |
| RandomEvaluation Mode=N/A2026.03 | 14.36 | 23.71 | |
| Binary SearchEvaluation Mode=Offline, Method Paradigm=LLM-Based, Backbone=GPT-4o, Ground-truth Guidance=false2026.03 | 13.54 | 32.03 | |
| Step-by-StepEvaluation Mode=Offline, Method Paradigm=LLM-Based, Backbone=GPT-4o, Ground-truth Guidance=false2026.03 | 13.25 | 28.14 | |
| All-at-OnceEvaluation Mode=Offline, Method Paradigm=LLM-Based, Backbone=GPT-4o, Ground-truth Guidance=false2026.03 | 10.37 | 51.85 | |
| All-at-Once (G)Evaluation Mode=Offline, Method Paradigm=LLM-Based, Backbone=GPT-4o, Ground-truth Guidance=true2026.03 | 10.22 | 54.59 |