Dialog Workflow Extraction and Evaluation on SynthABCD 1.0 (test)
89.49Macro ScoreQA-CoT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| QA-CoTModel=o3-mini, Prompting Strategy=QA-CoT2025.02 | 89.49 | 90.21 | 10.31 | |
| QA-CoTModel=o1-mini, Prompting Strategy=QA-CoT2025.02 | 87.16 | 87.17 | 12.43 | |
| QA-CoTModel=o1, Prompting Strategy=QA-CoT2025.02 | 80.6 | 76.95 | 9.62 | |
| BasicModel=o3-mini, Prompting Strategy=Basic2025.02 | 79.5 | 78.91 | 9.14 | |
| BasicModel=o1-mini, Prompting Strategy=Basic2025.02 | 69.57 | 67.39 | 10.56 | |
| QA-CoTModel=DeepSeek R1, Prompting Strategy=QA-CoT2025.02 | 66.95 | 55.65 | 8.41 | |
| BasicModel=DeepSeek R1, Prompting Strategy=Basic2025.02 | 65.26 | 51.52 | 7.5 | |
| BasicModel=o1, Prompting Strategy=Basic2025.02 | 59.57 | 52.39 | 7.65 |