Agent task completion on ClawGym-Bench
76Product & Collaboration ScoreGPT-5.4
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| GPT-5.4Model Group=Proprietary Frontier Models, Context Length=64K2026.04 | 76 | 66.56 | 73.83 | 76.19 | 68.51 | 82.4 | 73.49 | |
| Claude-4.7-OpusModel Group=Proprietary Frontier Models, Context Length=64K2026.04 | 73.02 | 74.28 | 80.14 | 80.98 | 81.04 | 82.04 | 77.81 | |
| GLM-5.1Model Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 70.95 | 64.11 | 71.71 | 75.99 | 74.09 | 73.86 | 71.12 | |
| Claude-4.6-SonnetModel Group=Proprietary Frontier Models, Context Length=64K2026.04 | 70.44 | 68.04 | 76.27 | 74.59 | 74.21 | 73.4 | 72.4 | |
| Gemini-3-FlashModel Group=Proprietary Frontier Models, Context Length=64K2026.04 | 68.7 | 68.06 | 68.89 | 78.87 | 80.68 | 84.04 | 73.5 | |
| Claude-4.6-OpusModel Group=Proprietary Frontier Models, Context Length=64K2026.04 | 68.02 | 63.49 | 73.23 | 74.4 | 73.38 | 82.9 | 71.43 | |
| Qwen3.5-PlusModel Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 67.34 | 66.56 | 74.27 | 73.07 | 71.79 | 72.1 | 70.35 | |
| Kimi-K2.6Model Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 67.16 | 55.99 | 72.56 | 72.84 | 71.56 | 82.47 | 69.05 | |
| DeepSeek-V3.2Model Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 64.45 | 64.43 | 71.97 | 72.3 | 66.66 | 70.34 | 67.9 | |
| MiniMax-M2.7Model Group=Open-Weight Frontier Models, Context Length=64K2026.04 | 63.98 | 60.11 | 63.96 | 63.42 | 68.72 | 64.1 | 63.72 | |
| Qwen3-235A23BModel Group=Compact Open-Weight Models, Context Length=64K2026.04 | 53.66 | 52.27 | 47.18 | 61.39 | 67.33 | 49.23 | 54.48 | |
| ClawGym-30A3BModel Group=ClawGym-Agents, Context Length=64K2026.04 | 52.98 | 50.97 | 64.64 | 61.46 | 57.9 | 56.13 | 56.82 | |
| ClawGym-8BModel Group=ClawGym-Agents, Context Length=64K2026.04 | 49.47 | 46.83 | 46.35 | 55.37 | 52.29 | 54.78 | 50.24 | |
| ClawGym-4BModel Group=ClawGym-Agents, Context Length=64K2026.04 | 45.21 | 46.6 | 47.51 | 48.99 | 52.25 | 48.27 | 47.73 | |
| Qwen3-30A3BModel Group=Compact Open-Weight Models, Context Length=64K2026.04 | 42.47 | 42.09 | 45.04 | 51.95 | 45.98 | 46.24 | 45.11 | |
| Qwen3-32BModel Group=Compact Open-Weight Models, Context Length=64K2026.04 | 40.68 | 36.21 | 37.84 | 47.16 | 49.29 | 33.11 | 40.32 | |
| Qwen3-8BModel Group=Compact Open-Weight Models, Context Length=64K2026.04 | 37.46 | 29.06 | 30.4 | 41.12 | 44.47 | 30.69 | 35.02 |