Shallow Retrieval on MultiWOZ 2.2
98Vanilla APAVanilla
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| VanillaModel Pair (Architect / Executive)=DeepSeek Reasoner / V3.22026.04 | 98 | — | — | — | |
| VanillaModel Pair (Architect / Executive)=GPT 5.4 / 5.4 mini2026.04 | 70.9 | — | — | — | |
| VanillaModel Pair (Architect / Executive)=Gemini 3.1 Pro / 2.5 Flash2026.04 | 64 | — | — | — | |
| VanillaModel Pair (Architect / Executive)=Claude Sonnet 4.6 / Haiku 4.52026.04 | 42.2 | — | — | — | |
| SSRPModel Pair (Architect / Executive)=Gemini 3.1 Pro / 2.5 Flash2026.04 | — | 80.9 | 26.41 | 1.52 | |
| SSRPModel Pair (Architect / Executive)=Claude Sonnet 4.6 / Haiku 4.52026.04 | — | 43.9 | 4.03 | 5.35 | |
| SSRPModel Pair (Architect / Executive)=DeepSeek Reasoner / V3.22026.04 | — | 99 | 1.02 | 2.51 | |
| SSRPModel Pair (Architect / Executive)=GPT 5.4 / 5.4 mini2026.04 | — | 87.5 | 23.41 | 4.2 |