Agent Task Completion on PinchBench
88.7Pass@1Gemini-3-Flash
Evaluation Results
| Method | Links | |
|---|---|---|
| Gemini-3-FlashModel Group=Proprietary Frontier Models, Context Length=64K2026.04 | 88.7 | |
| ClawGym-30A3BModel Group=ClawGym-Agents, Context Length=64K2026.04 | 86 | |
| MiniMax-M2.7Model Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 80.9 | |
| Claude-4.7-OpusModel Group=Proprietary Frontier Models, Context Length=64K2026.04 | 79.4 | |
| Claude-4.6-SonnetModel Group=Proprietary Frontier Models, Context Length=64K2026.04 | 79.3 | |
| Qwen3.5-PlusModel Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 78.7 | |
| GLM-5.1Model Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 76.4 | |
| ClawGym-4BModel Group=ClawGym-Agents, Context Length=64K2026.04 | 76.4 | |
| ClawGym-8BModel Group=ClawGym-Agents, Context Length=64K2026.04 | 75.7 | |
| Claude-4.6-OpusModel Group=Proprietary Frontier Models, Context Length=64K2026.04 | 75.3 | |
| GPT-5.4Model Group=Proprietary Frontier Models, Context Length=64K2026.04 | 68.3 | |
| Kimi-K2.6Model Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 66.5 | |
| DeepSeek-V3.2Model Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 60.8 | |
| Qwen3-235A23BModel Group=Compact Open-Weight Models, Context Length=64K2026.04 | 60.6 | |
| Qwen3-30A3BModel Group=Compact Open-Weight Models, Context Length=64K2026.04 | 55.6 | |
| Qwen3-8BModel Group=Compact Open-Weight Models, Context Length=64K2026.04 | 54.5 | |
| Qwen3-32BModel Group=Compact Open-Weight Models, Context Length=64K2026.04 | 49.4 |