Multi-turn conversation performance on Database
96.3Avg PerformanceFull
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FullModel=GPT-5.2, Setting=Ideal instructions2026.02 | 96.3 | 95.9 | |
| FullModel=DeepSeek-v3.2-Thinking, Setting=Ideal instructions2026.02 | 94.4 | 88.8 | |
| FullModel=GPT-4o-mini, Setting=Ideal instructions2026.02 | 92.5 | 93.5 | |
| Experience-Driven MediatorModel=DeepSeek-v3.2-Thinking, Setting=Ambiguous user inputs with Mediator2026.02 | 67.3 | 55.9 | |
| Experience-Driven MediatorModel=GPT-4o-mini, Setting=Ambiguous user inputs with Mediator2026.02 | 65.3 | 59.8 | |
| Experience-Driven MediatorModel=GPT-5.2, Setting=Ambiguous user inputs with Mediator2026.02 | 64.5 | 56.7 | |
| ShardedModel=GPT-4o-mini, Setting=Ambiguous user inputs2026.02 | 52.5 | 54.2 | |
| ShardedModel=GPT-5.2, Setting=Ambiguous user inputs2026.02 | 49.4 | 48 | |
| ShardedModel=DeepSeek-v3.2-Thinking, Setting=Ambiguous user inputs2026.02 | 43.6 | 54.2 |