Multi-step tool use execution on M3ToolEval
88Success RateRubricRefine
Evaluation results
| Method | Links | |
|---|---|---|
| RubricRefineModel=Sonnet-4.62026.05 | 88 | |
| RubricRefineModel=GPT-4.1-mini2026.05 | 86 | |
| RubricRefineModel=GPT-4o2026.05 | 86 | |
| RubricRefineModel=o3-mini2026.05 | 85 | |
| RubricRefineModel=GPT-4.12026.05 | 85 | |
| RubricRefineModel=Gemma-4-26B2026.05 | 85 | |
| RubricRefineModel=Qwen3.6-27B2026.05 | 85 | |
| Fixed RubricRefineModel=Sonnet-4.62026.05 | 84 | |
| Fixed RubricRefineModel=GPT-4.1-mini2026.05 | 80 | |
| Fixed RubricRefineModel=GPT-4.12026.05 | 80 | |
| Fixed RubricRefineModel=o3-mini2026.05 | 78 | |
| BoN+rubricModel=o3-mini2026.05 | 77 | |
| BoN+rubricModel=GPT-4.12026.05 | 76 | |
| BoN+rubricModel=Sonnet-4.62026.05 | 76 | |
| Self-DebugModel=GPT-4.1-mini2026.05 | 75 | |
| BoN+rubricModel=GPT-4.1-mini2026.05 | 75 | |
| BoN+fixed rubricModel=GPT-4o2026.05 | 75 | |
| BoN+rubricModel=GPT-4o2026.05 | 75 | |
| Fixed RubricRefineModel=GPT-4o2026.05 | 75 | |
| Self-DebugModel=o3-mini2026.05 | 75 | |
| BoN+fixed rubricModel=o3-mini2026.05 | 75 | |
| Self-DebugModel=GPT-4.12026.05 | 75 | |
| BoN+fixed rubricModel=Sonnet-4.62026.05 | 75 | |
| BoN+fixed rubricModel=GPT-4.1-mini2026.05 | 74 | |
| Fixed RubricRefineModel=Qwen3.6-27B2026.05 | 74 | |
| CodeActModel=Sonnet-4.62026.05 | 74 | |
| Best-of-NModel=Sonnet-4.62026.05 | 74 | |
| Self-DebugModel=GPT-4o2026.05 | 73 | |
| BoN+fixed rubricModel=GPT-4.12026.05 | 73 | |
| Fixed RubricRefineModel=Gemma-4-26B2026.05 | 73 | |
| Self-RefineModel=Gemma-4-26B2026.05 | 71 | |
| Self-RefineModel=Sonnet-4.62026.05 | 69 | |
| CodeActModel=GPT-4o2026.05 | 67 | |
| CodeActModel=o3-mini2026.05 | 66 | |
| Best-of-NModel=GPT-4.1-mini2026.05 | 65 | |
| Best-of-NModel=GPT-4o2026.05 | 65 | |
| CodeActModel=GPT-4.12026.05 | 65 | |
| Self-DebugModel=Gemma-4-26B2026.05 | 65 | |
| Self-RefineModel=Qwen3.6-27B2026.05 | 65 | |
| CodeActModel=GPT-4.1-mini2026.05 | 64 | |
| Self-RefineModel=GPT-4o2026.05 | 62 | |
| Best-of-NModel=GPT-4.12026.05 | 62 | |
| Best-of-NModel=o3-mini2026.05 | 61 | |
| Self-RefineModel=GPT-4.1-mini2026.05 | 60 | |
| Self-RefineModel=o3-mini2026.05 | 60 | |
| Self-RefineModel=GPT-4.12026.05 | 60 | |
| Best-of-NModel=Gemma-4-26B2026.05 | 53 | |
| BoN+fixed rubricModel=Gemma-4-26B2026.05 | 52 | |
| Best-of-NModel=Qwen3.6-27B2026.05 | 51 | |
| BoN+fixed rubricModel=Qwen3.6-27B2026.05 | 51 | |
| BoN+rubricModel=Qwen3.6-27B2026.05 | 51 | |
| CodeActModel=Gemma-4-26B2026.05 | 50 | |
| BoN+rubricModel=Gemma-4-26B2026.05 | 50 | |
| CodeActModel=Qwen3.6-27B2026.05 | 47 |