Multi-turn conversation performance on Average
94.7Avg PerformanceFull
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FullModel=DeepSeek-v3.2-Thinking, Setting=Ideal instructions2026.02 | 94.7 | 88.7 | |
| FullModel=GPT-5.2, Setting=Ideal instructions2026.02 | 92.7 | 85.4 | |
| FullModel=GPT-4o-mini, Setting=Ideal instructions2026.02 | 86.9 | 83.2 | |
| Experience-Driven MediatorModel=DeepSeek-v3.2-Thinking, Setting=Ambiguous user inputs with Mediator2026.02 | 81.9 | 69.9 | |
| Experience-Driven MediatorModel=GPT-4o-mini, Setting=Ambiguous user inputs with Mediator2026.02 | 73.9 | 68.8 | |
| Experience-Driven MediatorModel=GPT-5.2, Setting=Ambiguous user inputs with Mediator2026.02 | 72.6 | 63.5 | |
| ShardedModel=DeepSeek-v3.2-Thinking, Setting=Ambiguous user inputs2026.02 | 60.8 | 56.2 | |
| ShardedModel=GPT-4o-mini, Setting=Ambiguous user inputs2026.02 | 53.6 | 54.2 | |
| ShardedModel=GPT-5.2, Setting=Ambiguous user inputs2026.02 | 48.5 | 46.8 |