Multi-turn Clarification on ClarifyMT-Bench
96.6Precision (P)Base Model
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Base ModelBackbone=Qwen-2.5-7B-It2025.12 | 96.6 | 36.5 | 60.3 | 42.4 | 38.8 | 72.9 | 57.9 | |
| Majority VotingBackbone=Qwen-2.5-7B-It2025.12 | 96.4 | 42.7 | 74.6 | 49.1 | 43.2 | 71.6 | 62.9 | |
| AT-CoTBackbone=Qwen-2.5-7B-It2025.12 | 93.9 | 73.2 | 69.7 | 54.7 | 50.8 | 43.7 | 64.3 | |
| CoTBackbone=Qwen-2.5-7B-It2025.12 | 89 | 55 | 63.1 | 65.6 | 62.7 | 63.1 | 66.4 | |
| ClarifyAgentBackbone=Qwen-2.5-7B-It2025.12 | 85.3 | 85.8 | 97.3 | 91.7 | 84 | 83.9 | 88 | |
| AT-CoTBackbone=Llama-3.1-8B-It2025.12 | 74.5 | 80.4 | 84.8 | 81.8 | 87 | 29.2 | 73 | |
| Intent-SimBackbone=Qwen-2.5-7B-It2025.12 | 65.2 | 59.4 | 60 | 60.8 | 61.4 | 50.1 | 59.5 | |
| ClarifyAgentBackbone=Llama-3.1-8B-It2025.12 | 58 | 97.2 | 100 | 97.1 | 96.5 | 81.7 | 88.4 | |
| Base ModelBackbone=Llama-3.1-8B-It2025.12 | 48.3 | 75.6 | 87.5 | 86.4 | 84.8 | 44.4 | 71.2 | |
| Majority VotingBackbone=Llama-3.1-8B-It2025.12 | 48.3 | 76.9 | 86.4 | 89 | 91.2 | 43.3 | 72.4 | |
| Intent-SimBackbone=Llama-3.1-8B-It2025.12 | 22.6 | 83.3 | 84.4 | 86.6 | 77.6 | 15 | 61.6 | |
| CoTBackbone=Llama-3.1-8B-It2025.12 | 12.7 | 80.2 | 89 | 88.6 | 90.1 | 27.3 | 64.7 |