Task Fulfillment on MCPBench
4.38Task Fulfillment ScoreKimi-K2 Thinking
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Kimi-K2 ThinkingExecution Strategy=All Tools Loaded2026.03 | 4.38 | 20 | 23,768 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Tool Loading, Programmatic Tool Calling (PTC)=true, Reinforcement Learning=RL w/ Qwen3-30B J. (Rubrics)2026.03 | 4.15 | 18 | 13,400 | |
| Kimi-K2 ThinkingExecution Strategy=Iterative Server Loading2026.03 | 4.11 | 27 | 21,747 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Tool Loading, Reinforcement Learning=RL w/ Qwen3-30B J. (Rubrics)2026.03 | 4.03 | 20 | 11,151 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Tool Loading, Programmatic Tool Calling (PTC)=true, Reinforcement Learning=RL w/ Qwen3-30B J. (Generic)2026.03 | 3.91 | 20 | 12,951 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Server Loading, Reinforcement Learning=RL w/ Qwen3-30B Judge (Rubrics)2026.03 | 3.87 | 19 | 11,142 | |
| Kimi-K2 ThinkingExecution Strategy=Iterative Tool Loading2026.03 | 3.62 | 28 | 18,290 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Tool Loading, Reinforcement Learning=RL w/ Qwen3-30B J. (Generic)2026.03 | 3.48 | 23 | 12,815 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Server Loading, Reinforcement Learning=RL w/ GPT-4o Judge (Rubrics)2026.03 | 3.43 | 18 | 10,654 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Server Loading, Reinforcement Learning=RL w/ GPT-4o Judge (Generic)2026.03 | 3.25 | 17 | 9,372 | |
| Qwen2.5-7B-InstructExecution Strategy=Iterative Server Loading, Reinforcement Learning=RL w/ Qwen3-30B Judge (Rubrics)2026.03 | 3.18 | 21 | 13,075 | |
| Qwen2.5-7B-InstructExecution Strategy=Iterative Server Loading, Reinforcement Learning=RL w/ GPT-4o Judge (Rubrics)2026.03 | 3.13 | 19 | 11,882 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Server Loading, Reinforcement Learning=RL w/ Qwen3-30B Judge (Generic)2026.03 | 3.12 | 21 | 11,530 | |
| Qwen2.5-7B-InstructExecution Strategy=Iterative Server Loading, Reinforcement Learning=RL w/ GPT-4o Judge (Generic)2026.03 | 3.02 | 19 | 10,307 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Tool Loading, Programmatic Tool Calling (PTC)=true2026.03 | 2.94 | 18 | 13,462 | |
| Qwen2.5-7B-InstructExecution Strategy=Iterative Server Loading, Reinforcement Learning=RL w/ Qwen3-30B Judge (Generic)2026.03 | 2.84 | 24 | 12,201 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Server Loading2026.03 | 2.73 | 24 | 9,152 | |
| Qwen3-4B-InstructExecution Strategy=Iterative Tool Loading2026.03 | 2.36 | 20 | 9,045 | |
| Qwen2.5-7B-InstructExecution Strategy=Iterative Server Loading2026.03 | 2.33 | 25 | 11,192 |