Agent on τ2-Bench
85.4AccuracyGemini-3-Pro
Evaluation results
| Method | Links | |
|---|---|---|
| Gemini-3-Pro2026.03 | 85.4 | |
| Intern-S1-ProNumber of Parameters=1T-A22B2026.03 | 80.9 | |
| Kimi-K2.5Number of Parameters=1T-A32B2026.03 | 76.8 | |
| GPT-5.22026.03 | 76.6 | |
| JT-Safe-V2-35BParameters=35B2026.05 | 70 | |
| LongCat-Flash Exp-ChatEvaluation Mode=Chat2025.12 | 69.5 | |
| GLM 4.6Evaluation Mode=Chat2025.12 | 69.1 | |
| LongCat-Flash ChatEvaluation Mode=Chat2025.12 | 68.8 | |
| Lowest CentroidModel=GPT-OSS-120B, Number of Samples=642026.04 | 65 | |
| DeepSeek V3.2Evaluation Mode=Chat2025.12 | 64 | |
| Greedy DecodingModel=GPT-OSS-120B, Number of Samples=642026.04 | 58 | |
| Qwen3-VL-235B-ThinkingNumber of Parameters=235B-A22B, Thinking Configuration=true2026.03 | 57.4 | |
| Self-CertaintyModel=GPT-OSS-120B, Number of Samples=642026.04 | 57 | |
| Pass@1Model=GPT-OSS-120B, Number of Samples=642026.04 | 54.9 | |
| Tail ConfidenceModel=GPT-OSS-120B, Number of Samples=642026.04 | 53 | |
| Bottom WindowModel=GPT-OSS-120B, Number of Samples=642026.04 | 49 | |
| SOTA with Equivalent ParametersModel comparison=Equivalent Parameters2026.05 | 48.1 | |
| Lowest CentroidModel=Qwen3-14B, Number of Samples=642026.04 | 35 | |
| Greedy DecodingModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 33 | |
| Lowest CentroidModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 33 | |
| Greedy DecodingModel=Qwen3-14B, Number of Samples=642026.04 | 29 | |
| Pass@1Model=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 26.5 | |
| Pass@1Model=Qwen3-14B, Number of Samples=642026.04 | 25.9 | |
| Bottom WindowModel=Qwen3-14B, Number of Samples=642026.04 | 23 | |
| Self-CertaintyModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 22 | |
| Tail ConfidenceModel=Qwen3-14B, Number of Samples=642026.04 | 21 | |
| Tail ConfidenceModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 21 | |
| Bottom WindowModel=Ministral-3-14B-Instruct, Number of Samples=642026.04 | 21 | |
| Self-CertaintyModel=Qwen3-14B, Number of Samples=642026.04 | 18 | |
| Lowest CentroidModel=QWQ-32B, Number of Samples=642026.04 | 18 | |
| Pass@1Model=QWQ-32B, Number of Samples=642026.04 | 12.2 | |
| Lowest CentroidModel=Olmo-3.1-Think, Number of Samples=642026.04 | 12 | |
| Greedy DecodingModel=QWQ-32B, Number of Samples=642026.04 | 12 | |
| Pass@1Model=Olmo-3.1-Think, Number of Samples=642026.04 | 9.5 | |
| Self-CertaintyModel=QWQ-32B, Number of Samples=642026.04 | 9 | |
| Bottom WindowModel=QWQ-32B, Number of Samples=642026.04 | 9 | |
| Bottom WindowModel=Olmo-3.1-Think, Number of Samples=642026.04 | 8 | |
| Tail ConfidenceModel=QWQ-32B, Number of Samples=642026.04 | 8 | |
| Greedy DecodingModel=Olmo-3.1-Think, Number of Samples=642026.04 | 7 | |
| Self-CertaintyModel=Olmo-3.1-Think, Number of Samples=642026.04 | 7 | |
| Tail ConfidenceModel=Olmo-3.1-Think, Number of Samples=642026.04 | 4 |