Workflow Extraction on SynthABCD
91.85Macro ScoreReflect
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ReflectBackbone=Opus2025.02 | 91.85 | 92.82 | 12.47 | |
| BasicBackbone=Opus2025.02 | 91.43 | 91.52 | 12.29 | |
| QA-CoTBackbone=Opus2025.02 | 89.73 | 90.21 | 12.27 | |
| QA-CoTBackbone=Sonnet2025.02 | 88.38 | 86.73 | 10.46 | |
| QA-CoT + ReflectBackbone=GPT-4o2025.02 | 87.87 | 89.13 | 12.28 | |
| QA-CoT + ReflectBackbone=Opus2025.02 | 87.12 | 85.21 | 11.83 | |
| QA-CoTBackbone=GPT-4o2025.02 | 86.53 | 86.08 | 11.96 | |
| EnsembleBackbone=GPT-4o2025.02 | 84.31 | 81.3 | 9.62 | |
| QA-CoTBackbone=Gemini2025.02 | 84.04 | 81.96 | 9.7 | |
| QA-CoT + ReflectBackbone=Sonnet2025.02 | 82.59 | 85.65 | 11.08 | |
| EnsembleBackbone=Gemini2025.02 | 81.13 | 79.78 | 10.41 | |
| EnsembleBackbone=Opus2025.02 | 78.7 | 73.26 | 9.21 | |
| PlanBackbone=GPT-4o2025.02 | 78.4 | 68.91 | 8.55 | |
| QA-CoT + ReflectBackbone=Gemini2025.02 | 76.59 | 73.91 | 9.89 | |
| PlanBackbone=Opus2025.02 | 74.85 | 70.21 | 9.07 | |
| BasicBackbone=Gemini2025.02 | 74.73 | 75.65 | 8.59 | |
| PlanBackbone=Sonnet2025.02 | 73.02 | 70.65 | 9.32 | |
| PlanBackbone=Gemini2025.02 | 72.77 | 70.65 | 7.97 | |
| BasicBackbone=GPT-4o2025.02 | 68.91 | 61.95 | 8.71 | |
| ReflectBackbone=Gemini2025.02 | 61.9 | 60.43 | 9.08 | |
| ReflectBackbone=GPT-4o2025.02 | 57.42 | 52.61 | 8.24 | |
| EnsembleBackbone=Sonnet2025.02 | 55.97 | 45.43 | 7.99 | |
| BasicBackbone=Sonnet2025.02 | 51.23 | 45.43 | 7.49 | |
| ReflectBackbone=Sonnet2025.02 | 49.14 | 41.08 | 7.68 |