Tool-use on MCP-Atlas
65.2Pass RateGround Truth Server
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Ground Truth ServerSetting=Default2026.06 | 65.2 | 71 | |
| SINGSetting=Restricted2026.06 | 49.6 | 59.5 | |
| MCP-ZeroSetting=Restricted2026.06 | 45.4 | 56.7 | |
| SINGSetting=Global2026.06 | 40.2 | 51 | |
| MCP-ZeroSetting=Global2026.06 | 34 | 47.1 | |
| EnvFactoryBackbone=Qwen3-8B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 13.75 | 25.98 | |
| EnvScalerBackbone=Qwen3-4B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 9.97 | 22.27 | |
| EnvFactoryBackbone=Qwen3-4B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 9.97 | 21.89 | |
| EnvScalerBackbone=Qwen3-8B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 9.62 | 22.63 | |
| EnvFactory (SFT)Backbone=Qwen3-8B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 8.25 | 22.86 | |
| EnvFactory (SFT)Backbone=Qwen3-4B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 7.9 | 19.66 | |
| AWMBackbone=Qwen3-8B, Env. (Data Scale)=526, Tasks (Data Scale)=3,3152026.05 | 6.19 | 16.6 | |
| BaseBackbone=Qwen3-8B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 5.15 | 14.86 | |
| AWMBackbone=Qwen3-4B, Env. (Data Scale)=526, Tasks (Data Scale)=3,3152026.05 | 4.47 | 12.33 | |
| BaseBackbone=Qwen3-4B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 4.12 | 12.86 | |
| EnvFactoryBackbone=Qwen3-1.7B, Env. (Data Scale)=85, Tasks (Data Scale)=2,5752026.05 | 3.09 | 9.64 | |
| EnvScalerBackbone=Qwen3-1.7B, Env. (Data Scale)=191, Tasks (Data Scale)=11,5722026.05 | 2.75 | 9.4 | |
| EnvFactory (SFT)Backbone=Qwen3-1.7B, Env. (Data Scale)=85, Tasks (Data Scale)=1,6222026.05 | 1.72 | 10.05 | |
| BaseBackbone=Qwen3-1.7B, Env. (Data Scale)=–, Tasks (Data Scale)=–2026.05 | 1.03 | 6.25 |