# Iterative Self-Optimization on OPT-BENCH-ML 20 steps 1.0 Source: [Iterative Self-Optimization on OPT-BENCH-ML 20 steps 1.0]() Collection index: [llms.txt]() Data updated: 2026-05-12T10:11:26.070Z Evaluation of models performing Iterative Self-Optimization within the OPT-BENCH ML environment configured for 20 steps, measured by success rates (Win Count) and error rates (Buggy Rate, EG). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Iterative Self-Optimization | | Dataset (including any stated split) | OPT-BENCH-ML 20 steps 1.0 | | Metric | Win Count | | Unit | SCALAR | | Direction | Higher is better | | Results loaded | 19 of 19 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Win Count](): Higher is better; unit: SCALAR. - [Buggy Rate (w)](): Lower is better; unit: PERCENT. - [Buggy Rate (w/o)](): Lower is better; unit: PERCENT. - [EG (w)](): Lower is better; unit: PERCENT. - [EG (w/o)](): Lower is better; unit: PERCENT. ## Recorded results | Method | Win Count | Result date | Report paper | | --- | --- | --- | --- | | Qwen2.5-3B-Instruct | 8 | Not recorded | 6a02b409dbc836b87d824631 | | gemini-2.0-flash | 7 | Not recorded | 6a02b409dbc836b87d824631 | | claude-3-5-sonnet-20241022 | 7 | Not recorded | 6a02b409dbc836b87d824631 | | claude-3-7-sonnet-20250219 | 7 | Not recorded | 6a02b409dbc836b87d824631 | | grok-3 | 6 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-235B-Instruct | 6 | Not recorded | 6a02b409dbc836b87d824631 | | Internlm3-8b-Instruct | 6 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-7B-Instruct | 6 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-14B-Instruct | 6 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-32B-Instruct | 6 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-30B-A3B | 6 | Not recorded | 6a02b409dbc836b87d824631 | | gpt-o3-mini | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Deepseek-V3.1-Thinking | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-235B-Thinking | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-8B | 5 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen3-32B | 5 | Not recorded | 6a02b409dbc836b87d824631 | | gpt-4o-2024-08-06 | 4 | Not recorded | 6a02b409dbc836b87d824631 | | gpt-4.1-2025-04-14 | 4 | Not recorded | 6a02b409dbc836b87d824631 | | Qwen2.5-72B-Instruct | 4 | Not recorded | 6a02b409dbc836b87d824631 | ### Qwen2.5-3B-Instruct Win Count: 8. Result ID: 6a02fcc947f34ce924e653b2. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### gemini-2.0-flash Win Count: 7. Result ID: 6a02fcc947f34ce924e653aa. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### claude-3-5-sonnet-20241022 Win Count: 7. Result ID: 6a02fcc947f34ce924e653ab. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### claude-3-7-sonnet-20250219 Win Count: 7. Result ID: 6a02fcc947f34ce924e653ac. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### grok-3 Win Count: 6. Result ID: 6a02fcc947f34ce924e653ad. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen3-235B-Instruct Win Count: 6. Result ID: 6a02fcc947f34ce924e653b0. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### Internlm3-8b-Instruct Win Count: 6. Result ID: 6a02fcc947f34ce924e653b1. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen2.5-7B-Instruct Win Count: 6. Result ID: 6a02fcc947f34ce924e653b3. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen2.5-14B-Instruct Win Count: 6. Result ID: 6a02fcc947f34ce924e653b4. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen2.5-32B-Instruct Win Count: 6. Result ID: 6a02fcc947f34ce924e653b5. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen3-30B-A3B Win Count: 6. Result ID: 6a02fcc947f34ce924e653b8. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### gpt-o3-mini Win Count: 5. Result ID: 6a02fcc947f34ce924e653a9. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"Reasoning"}. This record does not assert independent reproduction. ### Deepseek-V3.1-Thinking Win Count: 5. Result ID: 6a02fcc947f34ce924e653ae. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"Reasoning"}. This record does not assert independent reproduction. ### Qwen3-235B-Thinking Win Count: 5. Result ID: 6a02fcc947f34ce924e653af. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"Reasoning"}. This record does not assert independent reproduction. ### Qwen3-8B Win Count: 5. Result ID: 6a02fcc947f34ce924e653b7. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen3-32B Win Count: 5. Result ID: 6a02fcc947f34ce924e653b9. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. ### gpt-4o-2024-08-06 Win Count: 4. Result ID: 6a02fcc947f34ce924e653a7. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### gpt-4.1-2025-04-14 Win Count: 4. Result ID: 6a02fcc947f34ce924e653a8. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Proprietary","Model Type":"General"}. This record does not assert independent reproduction. ### Qwen2.5-72B-Instruct Win Count: 4. Result ID: 6a02fcc947f34ce924e653b6. [Reporting paper]() Recorded evaluation configuration: {"Steps":"20","Model Category":"Open-Source","Model Type":"General"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.