# Iterative Self-Optimization on OPT-BENCH ML environments 10 steps 1.0 Source: [Iterative Self-Optimization on OPT-BENCH ML environments 10 steps 1.0]() Collection index: [llms.txt]() Data updated: 2026-05-12T10:11:26.032Z Evaluation of models performing Iterative Self-Optimization within the OPT-BENCH ML environments (version 1.0, 10 steps), measured by Win Count and Buggy Rates. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Iterative Self-Optimization | | Dataset (including any stated split) | OPT-BENCH ML environments 10 steps 1.0 | | Metric | Win Count | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Win Count](): Higher is better; unit: SCALAR. - [Buggy Rate (w)](): Lower is better; unit: PERCENT. - [Buggy Rate (w.o)](): Lower is better; unit: PERCENT. - [EG (w)](): Higher is better; unit: PERCENT. - [EG (w.o)](): Higher is better; unit: PERCENT. ## Recorded results | Method | Win Count | Result date | Report paper | | --- | --- | --- | --- | | claude-3-7-sonnet-20250219 | 8 | Not recorded | 6a02b409dbc836b87d824631 | | claude-3-5-sonnet-20241022 | 7 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-3B-Instruct | 7 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-7B-Instruct | 7 | Not recorded | 6a02b409dbc836b87d824631 | | gpt-o3-mini | 6 | Not recorded | 6a02b409dbc836b87d824631 | | gemini-2.0-flash | 6 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-235B-Instruct | 6 | Not recorded | 6a02b409dbc836b87d824631 | | Internlm3-8b-Instruct | 6 | Not recorded | 6a02b409dbc836b87d824631 | | grok-3 | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Deepseek-V3.1-Thinking | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-235B-Thinking | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-14B-Instruct | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-32B-Instruct | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-72B-Instruct | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-8B | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-30B-A3B | 5 | Not recorded | 6a02b409dbc836b87d824631 | | gpt-4o-2024-08-06 | 4 | Not recorded | 6a02b409dbc836b87d824631 | | gpt-4.1-2025-04-14 | 4 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-32B | 4 | Not recorded | 6a02b409dbc836b87d824631 | ### claude-3-7-sonnet-20250219 Win Count: 8. Result ID: 6a02fcc947f34ce924e65399. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### claude-3-5-sonnet-20241022 Win Count: 7. Result ID: 6a02fcc947f34ce924e65398. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen2.5-3B-Instruct Win Count: 7. Result ID: 6a02fcc947f34ce924e6539f. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Win Count: 7. Result ID: 6a02fcc947f34ce924e653a0. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### gpt-o3-mini Win Count: 6. Result ID: 6a02fcc947f34ce924e65396. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"Reasoning"}. This record does not assert independent reproduction. ### gemini-2.0-flash Win Count: 6. Result ID: 6a02fcc947f34ce924e65397. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen3-235B-Instruct Win Count: 6. Result ID: 6a02fcc947f34ce924e6539d. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### Internlm3-8b-Instruct Win Count: 6. Result ID: 6a02fcc947f34ce924e6539e. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### grok-3 Win Count: 5. Result ID: 6a02fcc947f34ce924e6539a. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### Deepseek-V3.1-Thinking Win Count: 5. Result ID: 6a02fcc947f34ce924e6539b. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"Reasoning"}. This record does not assert independent reproduction. ### Qwen3-235B-Thinking Win Count: 5. Result ID: 6a02fcc947f34ce924e6539c. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"Reasoning"}. This record does not assert independent reproduction. ### Qwen2.5-14B-Instruct Win Count: 5. Result ID: 6a02fcc947f34ce924e653a1. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen2.5-32B-Instruct Win Count: 5. Result ID: 6a02fcc947f34ce924e653a2. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen2.5-72B-Instruct Win Count: 5. Result ID: 6a02fcc947f34ce924e653a3. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen3-8B Win Count: 5. Result ID: 6a02fcc947f34ce924e653a4. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B Win Count: 5. Result ID: 6a02fcc947f34ce924e653a5. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### gpt-4o-2024-08-06 Win Count: 4. Result ID: 6a02fcc947f34ce924e65394. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### gpt-4.1-2025-04-14 Win Count: 4. Result ID: 6a02fcc947f34ce924e65395. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen3-32B Win Count: 4. Result ID: 6a02fcc947f34ce924e653a6. [Reporting paper]() Recorded evaluation configuration: {"Steps":"10","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.