Machine Learning Engineering on MLE-Bench full official
80.3Medal Rate (Low)MLEvolve
Evaluation Results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MLEvolveTime (h)=12, Backbone=Gemini-3.1-Pro-preview, Code Availability=Open-Source2026.06 | 80.3 | 64 | 46.7 | 65.3 | 100 | 76 | 34.7 | |
| MARS+Time (h)=24, Backbone=Gemini-3-Pro-preview, Code Availability=Proprietary2026.06 | 78.8 | 60.5 | 44.4 | 62.7 | 100 | 74.2 | 33.8 | |
| AIBuildAITime (h)=24, Backbone=Claude-Opus-4.6, Code Availability=Proprietary2026.06 | 77.3 | 61.4 | 46.7 | 63.1 | 100 | 71.1 | 25.8 | |
| ML-Master 2.0Time (h)=24, Backbone=DeepSeek-V3.2-Speciale, Code Availability=Open-Source2026.06 | 75.8 | 50.9 | 42.2 | 56.4 | 95.6 | 63.1 | 19.6 | |
| MARSTime (h)=24, Backbone=Gemini-3-Pro-preview, Code Availability=Proprietary2026.06 | 74.2 | 52.6 | 37.8 | 56 | 98.7 | 65.8 | 31.1 | |
| Gome (GPT-5)Time=12h, GPU=V1002026.03 | 68.2 | 21.1 | 22.2 | 35.1 | 96 | 45.3 | 16.4 | |
| MLE-STAR-Pro-1.5Time (h)=24, Backbone=Gemini-2.5-Pro, Code Availability=Proprietary2026.06 | 68.2 | 34.2 | 33.3 | 44 | 93.8 | 52.9 | 19.1 | |
| R&D-AgentTime (h)=12, Backbone=gpt-5, Code Availability=Open-Source2026.06 | 68.2 | 21.1 | 22.2 | 35.1 | 53.3 | 40.4 | 16.4 | |
| LeerooTime (h)=24, Backbone=Gemini-3-Pro-preview, Code Availability=Open-Source2026.06 | 68.2 | 44.7 | 40 | 50.7 | 50.7 | 50.7 | 21.3 | |
| FM-AgentTime (h)=24, Backbone=Gemini-2.5-Pro, Code Availability=Proprietary2026.06 | 62.1 | 36.8 | 33.3 | 43.6 | 96.9 | 51.6 | 22.7 | |
| Gome (o3)Time=12h, GPU=V1002026.03 | 59.1 | 20.2 | 22.2 | 32.5 | 94.2 | 43.1 | 17.3 | |
| AIRA (o3)Time=24h, GPU=H2002026.03 | 55 | 22 | 21.7 | 31.6 | 97.5 | 45.5 | 17.3 | |
| AIRA-DojoTime (h)=24, Backbone=o3, Code Availability=Open-Source2026.06 | 55 | 22 | 21.7 | 31.6 | 97.5 | 45.5 | 17.3 | |
| ML-Master (GPT-5)Time=12h, GPU=V1002026.03 | 51.5 | 10.5 | 17.8 | 24 | 98.2 | 34.7 | 12.4 | |
| Gome (DeepSeek-R1)Time=12h, GPU=V1002026.03 | 50 | 9.7 | 20 | 23.4 | 88.9 | 35.1 | 12.4 | |
| ML-MasterTime (h)=12, Backbone=DeepSeek-R1, Code Availability=Open-Source2026.06 | 48.5 | 20.2 | 24.4 | 29.3 | 93.3 | 44.9 | 17.3 | |
| ML-Master (DeepSeek-R1)Time=12h, GPU=V1002026.03 | 47 | 9.7 | 20 | 22.7 | 91.6 | 32.9 | 12.4 | |
| ML-Master (o3)Time=12h, GPU=V1002026.03 | 42.4 | 12.3 | 20 | 22.7 | 94.2 | 31.5 | 13.3 | |
| AIDETime (h)=24, Backbone=o1-preview, Code Availability=Open-Source2026.06 | 35.9 | 8.5 | 11.7 | 17.1 | 82.8 | 29.4 | 9.4 | |
| AIDE (o1-preview)Time=24h, GPU=A10†2026.03 | 34.3 | 8.8 | 10 | 16.9 | 82.8 | 29.4 | 9.4 | |
| AIDE (gpt-4o-24-08)Time=24h, GPU=A10†2026.03 | 19 | 3.2 | 5.6 | 8.6 | 54.9 | 14.4 | 5 | |
| OpenHands (gpt-4o-24-08)Time=24h, GPU=A10†2026.03 | 11.5 | 2.2 | 1.9 | 5.1 | 52 | 7.1 | 2.7 | |
| MLAB (gpt-4o-24-08)Time=24h, GPU=A10†2026.03 | 4.2 | 0 | 0 | 1.3 | 44.3 | 1.9 | 0.8 |