Interactive Tool-Use Agent Performance on VitaBench
65Delivery ScoreQwen3-SE-32B
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Qwen3-SE-32BEvaluation Protocol=Zero-shot, Sampling=Pass@42026.02 | 65 | 29 | 66 | 27 | 46.8 | |
| Qwen3-32BEvaluation Protocol=Zero-shot, Sampling=Pass@42026.02 | 57 | 15 | 58 | 14 | 36 | |
| Qwen3-SE-8BEvaluation Protocol=Zero-shot, Sampling=Pass@42026.02 | 56 | 12 | 57 | 18 | 35.8 | |
| GPT-5-thinkingModel Source=Closed-Source Large Language Models2025.12 | 54 | 22.8 | 52.5 | 37.5 | — | |
| GPT-o3Model Source=Closed-Source Large Language Models2025.12 | 53.5 | 30 | 53.5 | 37.8 | — | |
| Gemini-2.5-proModel Source=Closed-Source Large Language Models2025.12 | 49 | 23.5 | 43.8 | 26.5 | — | |
| Claude-Sonnet-4Model Source=Closed-Source Large Language Models2025.12 | 46 | 23 | 51.5 | 29 | — | |
| AutoForge-30B-A3BModel Source=Open-Source Large Language Models2025.12 | 46 | 17.5 | 54.5 | 24 | — | |
| Qwen3-235B-A22B-2507zero-shot=true2026.02 | 45 | 14.5 | 32 | 15.8 | — | |
| GPT-o4-miniModel Source=Closed-Source Large Language Models2025.12 | 44.5 | 19.5 | 46.5 | 23.5 | — | |
| Qwen3-Thinking-235B-A22BModel Source=Open-Source Large Language Models2025.12 | 44 | 18.8 | 46 | 17.5 | — | |
| Qwen3-8BEvaluation Protocol=Zero-shot, Sampling=Pass@42026.02 | 43 | 6 | 44 | 15 | 27 | |
| GPT-OSS-120B-A5Bzero-shot=true2026.02 | 37 | 15 | 42 | 12 | — | |
| GPT-OSS-120B-A5BModel Source=Open-Source Large Language Models2025.12 | 37 | 15 | 42 | 12 | — | |
| Kimi-K2-1T-A32BModel Source=Open-Source Large Language Models2025.12 | 35.3 | 15.5 | 42.5 | 22 | — | |
| Qwen3-Thinking-30B-A3BModel Source=Open-Source Large Language Models2025.12 | 35 | 16 | 40 | 20.5 | — | |
| Deepseek-V3.1-671B-A37BModel Source=Open-Source Large Language Models2025.12 | 34 | 16.3 | 42.5 | 18.3 | — | |
| Kimi-K2-0905zero-shot=true2026.02 | 32.5 | 11.5 | 30 | 18.8 | — | |
| Qwen3-SE-32Bzero-shot=true2026.02 | 31.3 | 10.8 | 34.5 | 12.5 | — | |
| MUA-RL-32BModel Source=Open-Source Large Language Models2025.12 | 30 | 10.5 | 32.5 | 11.5 | — | |
| Qwen3-32Bzero-shot=true2026.02 | 27 | 5.3 | 22.5 | 4.5 | — | |
| Qwen3-SE-8Bzero-shot=true2026.02 | 26.3 | 3 | 23.8 | 7 | — | |
| Seed-OSS-36Bzero-shot=true2026.02 | 26 | 6.1 | 39 | 7 | — | |
| xLAM-2-32B-fc-rzero-shot=true2026.02 | 26 | 4 | 17 | 10 | — | |
| xLAM-2-32B-fc-rModel Source=Open-Source Large Language Models2025.12 | 26 | 4 | 17 | 10 | — | |
| Seed-OSS-36BModel Source=Open-Source Large Language Models2025.12 | 26 | 6.1 | 39 | 7 | — | |
| AgentScaler-30B-A3BModel Source=Open-Source Large Language Models2025.12 | 25 | 8 | 33 | 16 | — | |
| EnvScalerBackbone=Qwen3-8B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 25 | — | 19 | 12 | 18.67 | |
| BaseBackbone=Qwen3-8B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 24 | — | 15 | 11.11 | 16.7 | |
| EnvFactoryBackbone=Qwen3-8B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 24 | — | 22 | 10 | 18.67 | |
| Qwen-Coder-30B-A3BModel Source=Open-Source Large Language Models2025.12 | 23 | 4 | 21 | 12 | — | |
| EnvScalerBackbone=Qwen3-4B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 23 | — | 15 | 6.06 | 14.69 | |
| EnvFactory (SFT)Backbone=Qwen3-8B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 23 | — | 20 | 7 | 16.67 | |
| AWMBackbone=Qwen3-4B, Env. (Data Scale)=526, Tasks (Data Scale)=3,3152026.05 | 22 | — | 13 | 0 | 11.67 | |
| EnvFactoryBackbone=Qwen3-4B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 21 | — | 21 | 6 | 16 | |
| AWMBackbone=Qwen3-8B, Env. (Data Scale)=526, Tasks (Data Scale)=3,3152026.05 | 20 | — | 15 | 14.43 | 16.48 | |
| EnvFactory (SFT)Backbone=Qwen3-4B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 19 | — | 12 | 3 | 11.33 | |
| Qwen3-8Bzero-shot=true2026.02 | 18.3 | 1.5 | 14.8 | 4.5 | — | |
| xLAM-2-70B-fc-rModel Source=Open-Source Large Language Models2025.12 | 14 | 2 | 11 | 2 | — | |
| EnvFactory (SFT)Backbone=Qwen3-1.7B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 13 | — | 6 | 0 | 6.33 | |
| EnvFactoryBackbone=Qwen3-1.7B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 11 | — | 11 | 0 | 7.33 | |
| EnvScalerBackbone=Qwen3-1.7B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 9 | — | 3 | 1.09 | 4.36 | |
| BaseBackbone=Qwen3-4B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 9 | — | 12 | 2.02 | 7.67 | |
| BaseBackbone=Qwen3-1.7B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 4 | — | 0 | 0 | 1.33 | |
| Gemini 2.5 Flash-Lite2026.01 | — | — | — | — | 4.5 | |
| LongCat-Flash-LiteArchitecture=MoE + NE, # Total Params=68.5B, # Activated Params=2.9B~4.5B2026.01 | — | — | — | — | 7 | |
| LongCat-Next2026.03 | — | — | — | — | 5.8 | |
| Qwen3-Next-80B-A3B-InstructArchitecture=MoE, # Total Params=80B, # Activated Params=3B2026.01 | — | — | — | — | 5.8 | |
| Qwen3-Next-80B-A3B-Instruct2026.03 | — | — | — | — | 5.8 |