Agentic Task Performance on τ2-Bench Airline 1.0 (test)
96.4CAPNOD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| NODBackbone=GPT-OSS-20B2026.05 | 96.4 | 47.3 | |
| REVISE-onlyBackbone=Qwen3-32B2026.05 | 96 | 48.7 | |
| NODBackbone=Qwen3-30B-MoE2026.05 | 96 | 52 | |
| Self-aware AbstentionBackbone=GPT-OSS-20B2026.05 | 95.3 | 38 | |
| Weak DirectorBackbone=GPT-OSS-20B2026.05 | 95.3 | 43.3 | |
| Self-ReflectionBackbone=GPT-OSS-20B2026.05 | 94.7 | 46 | |
| Self-aware AbstentionBackbone=Qwen3-32B2026.05 | 93.7 | 39.3 | |
| Weak DirectorBackbone=Ministral-3-14B2026.05 | 92.6 | 37.3 | |
| Vanilla LLMBackbone=GPT-OSS-20B2026.05 | 91.9 | 43.3 | |
| Self-aware AbstentionBackbone=Qwen3-30B-MoE2026.05 | 91.7 | 43.3 | |
| REVISE-onlyBackbone=Qwen3-30B-MoE2026.05 | 91.4 | 50.7 | |
| Vanilla LLMBackbone=Qwen3-30B-MoE2026.05 | 91.2 | 42.7 | |
| NODBackbone=Qwen3-14B2026.05 | 90.7 | 43.3 | |
| REVISE-onlyBackbone=GPT-OSS-20B2026.05 | 90.3 | 42 | |
| NODBackbone=Qwen3-32B2026.05 | 90.3 | 42.7 | |
| Weak DirectorBackbone=Qwen3-32B2026.05 | 90 | 40 | |
| Weak DirectorBackbone=Qwen3-30B-MoE2026.05 | 89.7 | 48 | |
| NODBackbone=Ministral-3-14B2026.05 | 89.3 | 40.7 | |
| AutoGenBackbone=Qwen3-32B2026.05 | 89.1 | 40.7 | |
| NODBackbone=Qwen3-8B2026.05 | 89 | 36.7 | |
| AutoGenBackbone=GPT-OSS-20B2026.05 | 88.8 | 44.7 | |
| REVISE-onlyBackbone=Ministral-3-14B2026.05 | 88.8 | 42 | |
| Multi-Agent DebateBackbone=Qwen3-8B2026.05 | 88.7 | 35.3 | |
| Multi-Agent DebateBackbone=GPT-OSS-20B2026.05 | 88.3 | 40.7 | |
| REVISE-onlyBackbone=Qwen3-8B2026.05 | 87.2 | 42.7 | |
| Multi-Agent DebateBackbone=Qwen3-30B-MoE2026.05 | 87 | 48 | |
| Weak DirectorBackbone=Qwen3-8B2026.05 | 87 | 35.3 | |
| AutoGenBackbone=Qwen3-30B-MoE2026.05 | 86.9 | 42.7 | |
| Weak DirectorBackbone=Qwen3-14B2026.05 | 86 | 36 | |
| REVISE-onlyBackbone=Qwen3-14B2026.05 | 85.9 | 44 | |
| AutoGenBackbone=Qwen3-14B2026.05 | 85.4 | 37.3 | |
| Self-ReflectionBackbone=Qwen3-14B2026.05 | 84.7 | 36 | |
| Self-aware AbstentionBackbone=Qwen3-8B2026.05 | 84.7 | 30.7 | |
| Self-ReflectionBackbone=Qwen3-30B-MoE2026.05 | 84.3 | 46 | |
| Self-ReflectionBackbone=Qwen3-32B2026.05 | 84.3 | 41.3 | |
| Multi-Agent DebateBackbone=Qwen3-32B2026.05 | 84.3 | 39.3 | |
| Vanilla LLMBackbone=Qwen3-32B2026.05 | 83.8 | 36.7 | |
| Self-ReflectionBackbone=Qwen3-8B2026.05 | 83.6 | 34 | |
| Self-aware AbstentionBackbone=Qwen3-14B2026.05 | 83.4 | 37.3 | |
| AutoGenBackbone=Qwen3-8B2026.05 | 82.3 | 31.3 | |
| Self-aware AbstentionBackbone=Ministral-3-14B2026.05 | 78.3 | 32.7 | |
| Multi-Agent DebateBackbone=Qwen3-14B2026.05 | 78.3 | 34 | |
| AutoGenBackbone=Ministral-3-14B2026.05 | 77.2 | 29.3 | |
| Vanilla LLMBackbone=Qwen3-8B2026.05 | 76.7 | 30 | |
| Vanilla LLMBackbone=Qwen3-14B2026.05 | 76.2 | 31.3 | |
| Self-ReflectionBackbone=Ministral-3-14B2026.05 | 64.2 | 22 | |
| Multi-Agent DebateBackbone=Ministral-3-14B2026.05 | 63.9 | 18.7 | |
| Vanilla LLMBackbone=Ministral-3-14B2026.05 | 63.5 | 21.3 |