Agentic Task Performance on τ2-Bench Retail 1.0 (test)
91.4Completion Accuracy (CAP)NOD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NODBackbone=Qwen3-30B-MoE2026.05 | 91.4 | 64.3 | |
| Weak DirectorBackbone=Ministral-3-14B2026.05 | 90.6 | 33.9 | |
| NODBackbone=GPT-OSS-20B2026.05 | 90.3 | 61.7 | |
| NODBackbone=Qwen3-32B2026.05 | 89.1 | 58.8 | |
| NODBackbone=Ministral-3-14B2026.05 | 89.1 | 54.4 | |
| Self-aware AbstentionBackbone=GPT-OSS-20B2026.05 | 89 | 14.3 | |
| Self-aware AbstentionBackbone=Qwen3-32B2026.05 | 88.3 | 29.5 | |
| Self-aware AbstentionBackbone=Ministral-3-14B2026.05 | 88.3 | 33.6 | |
| REVISE-onlyBackbone=GPT-OSS-20B2026.05 | 87 | 59.1 | |
| Vanilla LLMBackbone=Qwen3-30B-MoE2026.05 | 85.5 | 45.3 | |
| NODBackbone=Qwen3-8B2026.05 | 85.5 | 42.1 | |
| Multi-Agent DebateBackbone=Qwen3-30B-MoE2026.05 | 85.3 | 46.7 | |
| REVISE-onlyBackbone=Ministral-3-14B2026.05 | 85.2 | 52 | |
| REVISE-onlyBackbone=Qwen3-30B-MoE2026.05 | 84.7 | 64.6 | |
| Self-ReflectionBackbone=Qwen3-30B-MoE2026.05 | 84 | 46 | |
| Multi-Agent DebateBackbone=GPT-OSS-20B2026.05 | 84 | 37.1 | |
| Weak DirectorBackbone=GPT-OSS-20B2026.05 | 84 | 55.3 | |
| Self-ReflectionBackbone=GPT-OSS-20B2026.05 | 83.9 | 36.3 | |
| REVISE-onlyBackbone=Qwen3-14B2026.05 | 83.6 | 51.2 | |
| NODBackbone=Qwen3-14B2026.05 | 83.6 | 48.2 | |
| Self-aware AbstentionBackbone=Qwen3-30B-MoE2026.05 | 83.5 | 44.4 | |
| AutoGenBackbone=Qwen3-30B-MoE2026.05 | 83.5 | 45.3 | |
| Self-aware AbstentionBackbone=Qwen3-8B2026.05 | 83.4 | 20.5 | |
| Vanilla LLMBackbone=GPT-OSS-20B2026.05 | 83.2 | 38 | |
| REVISE-onlyBackbone=Qwen3-32B2026.05 | 82.8 | 58.5 | |
| Weak DirectorBackbone=Qwen3-30B-MoE2026.05 | 82.4 | 58.2 | |
| Weak DirectorBackbone=Qwen3-32B2026.05 | 81.7 | 52.9 | |
| AutoGenBackbone=GPT-OSS-20B2026.05 | 81.5 | 36.8 | |
| Multi-Agent DebateBackbone=Ministral-3-14B2026.05 | 81.4 | 36.8 | |
| Multi-Agent DebateBackbone=Qwen3-32B2026.05 | 81 | 41.4 | |
| AutoGenBackbone=Ministral-3-14B2026.05 | 80.9 | 36 | |
| Vanilla LLMBackbone=Ministral-3-14B2026.05 | 80.8 | 41.5 | |
| AutoGenBackbone=Qwen3-32B2026.05 | 79.9 | 43.6 | |
| Self-ReflectionBackbone=Ministral-3-14B2026.05 | 79.7 | 38.6 | |
| Self-ReflectionBackbone=Qwen3-32B2026.05 | 79.1 | 40.9 | |
| Weak DirectorBackbone=Qwen3-14B2026.05 | 79 | 47.4 | |
| Multi-Agent DebateBackbone=Qwen3-8B2026.05 | 78.8 | 32.6 | |
| Vanilla LLMBackbone=Qwen3-32B2026.05 | 78.5 | 42.6 | |
| Multi-Agent DebateBackbone=Qwen3-14B2026.05 | 76.7 | 38.1 | |
| Self-ReflectionBackbone=Qwen3-14B2026.05 | 76.2 | 36.1 | |
| AutoGenBackbone=Qwen3-14B2026.05 | 75.7 | 38.6 | |
| Self-aware AbstentionBackbone=Qwen3-14B2026.05 | 75.4 | 31.6 | |
| Self-ReflectionBackbone=Qwen3-8B2026.05 | 74.2 | 32.5 | |
| AutoGenBackbone=Qwen3-8B2026.05 | 74.2 | 32.8 | |
| Vanilla LLMBackbone=Qwen3-14B2026.05 | 73.9 | 40.7 | |
| Weak DirectorBackbone=Qwen3-8B2026.05 | 72.2 | 36.5 | |
| Vanilla LLMBackbone=Qwen3-8B2026.05 | 71.5 | 36.3 | |
| REVISE-onlyBackbone=Qwen3-8B2026.05 | 68 | 46.5 |