Dialog Workflow Extraction and Evaluation on ABCD 1.0 (test)
61.2Macro ScoreQA-CoT
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| QA-CoTModel=o3-mini, Prompting Strategy=QA-CoT2025.02 | 61.2 | 61.51 | 11.26 | |
| QA-CoTModel=o1-mini, Prompting Strategy=QA-CoT2025.02 | 60.74 | 64.74 | 12.05 | |
| BasicModel=o3-mini, Prompting Strategy=Basic2025.02 | 57.06 | 57.19 | 9.52 | |
| QA-CoTModel=DeepSeek R1, Prompting Strategy=QA-CoT2025.02 | 47.24 | 44.06 | 7.89 | |
| BasicModel=DeepSeek R1, Prompting Strategy=Basic2025.02 | 46.1 | 40.1 | 8.42 | |
| BasicModel=o1-mini, Prompting Strategy=Basic2025.02 | 44.2 | 40.46 | 9.73 | |
| QA-CoTModel=o1, Prompting Strategy=QA-CoT2025.02 | 39 | 33.09 | 8.5 | |
| BasicModel=o1, Prompting Strategy=Basic2025.02 | 36.72 | 35.97 | 7.7 |