# Task Fulfillment on MCPBench Source: [Task Fulfillment on MCPBench]() Collection index: [llms.txt]() Data updated: 2026-03-10T08:00:30.780Z Evaluation of Task Fulfillment capabilities on the MCPBench dataset, measuring performance via Task Fulfillment score, average turns, and average tokens consumed. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Task Fulfillment | | Dataset (including any stated split) | MCPBench | | Metric | Task Fulfillment Score | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Task Fulfillment Score](): Higher is better; unit: SCALAR. - [Average Turns](): Higher is better; unit: SCALAR. - [Average Tokens Consumed](): Lower is better; unit: SCALAR. ## Recorded results | Method | Task Fulfillment Score | Result date | Report paper | | --- | --- | --- | --- | | Kimi-K2 Thinking | 4.38 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 4.15 | Not recorded | 69afa5802d603e6de3e43a15 | | Kimi-K2 Thinking | 4.11 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 4.03 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 3.91 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 3.87 | Not recorded | 69afa5802d603e6de3e43a15 | | Kimi-K2 Thinking | 3.62 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 3.48 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 3.43 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 3.25 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen2.5-7B-Instruct | 3.18 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen2.5-7B-Instruct | 3.13 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 3.12 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen2.5-7B-Instruct | 3.02 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 2.94 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen2.5-7B-Instruct | 2.84 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 2.73 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen3-4B-Instruct | 2.36 | Not recorded | 69afa5802d603e6de3e43a15 | | Qwen2.5-7B-Instruct | 2.33 | Not recorded | 69afa5802d603e6de3e43a15 | ### Kimi-K2 Thinking Task Fulfillment Score: 4.38. Result ID: 69afceb56dbb7075b957ee4d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"All Tools Loaded"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 4.15. Result ID: 69afceb56dbb7075b957ee5f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Tool Loading","Programmatic Tool Calling (PTC)":"true","Reinforcement Learning":"RL w/ Qwen3-30B J. (Rubrics)"}. This record does not assert independent reproduction. ### Kimi-K2 Thinking Task Fulfillment Score: 4.11. Result ID: 69afceb56dbb7075b957ee4e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 4.03. Result ID: 69afceb56dbb7075b957ee5e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Tool Loading","Reinforcement Learning":"RL w/ Qwen3-30B J. (Rubrics)"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 3.91. Result ID: 69afceb56dbb7075b957ee5d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Tool Loading","Programmatic Tool Calling (PTC)":"true","Reinforcement Learning":"RL w/ Qwen3-30B J. (Generic)"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 3.87. Result ID: 69afceb56dbb7075b957ee59. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading","Reinforcement Learning":"RL w/ Qwen3-30B Judge (Rubrics)"}. This record does not assert independent reproduction. ### Kimi-K2 Thinking Task Fulfillment Score: 3.62. Result ID: 69afceb56dbb7075b957ee4f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Tool Loading"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 3.48. Result ID: 69afceb56dbb7075b957ee5c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Tool Loading","Reinforcement Learning":"RL w/ Qwen3-30B J. (Generic)"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 3.43. Result ID: 69afceb56dbb7075b957ee58. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading","Reinforcement Learning":"RL w/ GPT-4o Judge (Rubrics)"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 3.25. Result ID: 69afceb56dbb7075b957ee56. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading","Reinforcement Learning":"RL w/ GPT-4o Judge (Generic)"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Task Fulfillment Score: 3.18. Result ID: 69afceb56dbb7075b957ee54. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading","Reinforcement Learning":"RL w/ Qwen3-30B Judge (Rubrics)"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Task Fulfillment Score: 3.13. Result ID: 69afceb56dbb7075b957ee53. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading","Reinforcement Learning":"RL w/ GPT-4o Judge (Rubrics)"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 3.12. Result ID: 69afceb56dbb7075b957ee57. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading","Reinforcement Learning":"RL w/ Qwen3-30B Judge (Generic)"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Task Fulfillment Score: 3.02. Result ID: 69afceb56dbb7075b957ee51. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading","Reinforcement Learning":"RL w/ GPT-4o Judge (Generic)"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 2.94. Result ID: 69afceb56dbb7075b957ee5b. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Tool Loading","Programmatic Tool Calling (PTC)":"true"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Task Fulfillment Score: 2.84. Result ID: 69afceb56dbb7075b957ee52. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading","Reinforcement Learning":"RL w/ Qwen3-30B Judge (Generic)"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 2.73. Result ID: 69afceb56dbb7075b957ee55. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading"}. This record does not assert independent reproduction. ### Qwen3-4B-Instruct Task Fulfillment Score: 2.36. Result ID: 69afceb56dbb7075b957ee5a. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Tool Loading"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Task Fulfillment Score: 2.33. Result ID: 69afceb56dbb7075b957ee50. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Execution Strategy":"Iterative Server Loading"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.