Iterative Self-Optimization on OPT-BENCH ML environments 10 steps 1.0
8Win Countclaude-3-7-sonnet-20250219
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| claude-3-7-sonnet-20250219Steps=10, Model Category=Proprietary, Model Type=General2026.05 | 8 | 8 | 22 | 64 | 55 | |
| claude-3-5-sonnet-20241022Steps=10, Model Category=Proprietary, Model Type=General2026.05 | 7 | 18 | 30 | 51 | 46 | |
| Qwen2.5-3B-InstructSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 7 | 46 | 68 | 18 | 9 | |
| Qwen2.5-7B-InstructSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 7 | 38 | 54 | 31 | 23 | |
| gpt-o3-miniSteps=10, Model Category=Proprietary, Model Type=Reasoning2026.05 | 6 | 0 | 0 | 72 | 69 | |
| gemini-2.0-flashSteps=10, Model Category=Proprietary, Model Type=General2026.05 | 6 | 6 | 12 | 44 | 43 | |
| Qwen3-235B-InstructSteps=10, Model Category=Proprietary, Model Type=General2026.05 | 6 | 10 | 10 | 53 | 52 | |
| Internlm3-8b-InstructSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 6 | 30 | 42 | 20 | 18 | |
| grok-3Steps=10, Model Category=Proprietary, Model Type=General2026.05 | 5 | 14 | 12 | 57 | 56 | |
| Deepseek-V3.1-ThinkingSteps=10, Model Category=Proprietary, Model Type=Reasoning2026.05 | 5 | 0 | 0 | 78 | 75 | |
| Qwen3-235B-ThinkingSteps=10, Model Category=Proprietary, Model Type=Reasoning2026.05 | 5 | 0 | 0 | 75 | 76 | |
| Qwen2.5-14B-InstructSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 5 | 24 | 28 | 41 | 39 | |
| Qwen2.5-32B-InstructSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 5 | 30 | 28 | 41 | 43 | |
| Qwen2.5-72B-InstructSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 5 | 32 | 38 | 39 | 38 | |
| Qwen3-8BSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 5 | 20 | 20 | 57 | 58 | |
| Qwen3-30B-A3BSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 5 | 0 | 6 | 65 | 65 | |
| gpt-4o-2024-08-06Steps=10, Model Category=Proprietary, Model Type=General2026.05 | 4 | 26 | 30 | 44 | 44 | |
| gpt-4.1-2025-04-14Steps=10, Model Category=Proprietary, Model Type=General2026.05 | 4 | 8 | 8 | 47 | 48 | |
| Qwen3-32BSteps=10, Model Category=Open-Source, Model Type=General2026.05 | 4 | 10 | 10 | 57 | 59 |