ML Engineering on MLE-Bench official (test)
71.2Medal Rate (Low)Gome GPT-5 (24h)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Gome GPT-5 (24h)Framework=Gome, w/o Ext. Know.=true, Time=24h, GPU=V1002026.03 | 71.2 | 28.1 | 26.7 | 40.4 | |
| Gome GPT-5 (12h)Framework=Gome, w/o Ext. Know.=true, Time=12h, GPU=V1002026.03 | 68.2 | 21.1 | 22.2 | 35.1 | |
| Thesis GPT-5-CodexFramework=Thesis, w/o Ext. Know.=false, Time=24h, GPU=H1002026.03 | 65.2 | 45.6 | 31.1 | 48.4 | |
| MLE-STAR Gemini-2.5-proFramework=MLE-STAR, w/o Ext. Know.=false, Time=12h, GPU=8*V1002026.03 | 63.6 | — | — | — | |
| Gome o3 (24h)Framework=Gome, w/o Ext. Know.=true, Time=24h, GPU=V1002026.03 | 63.6 | 27.2 | 24.4 | 36.4 | |
| InternAgent-MLE DeepSeek-R1Framework=InternAgent-MLE, w/o Ext. Know.=false, Time=12h, GPU=A8002026.03 | 62.1 | 26.3 | 24.4 | 36.4 | |
| FM Agent Gemini-2.5-proFramework=FM Agent, w/o Ext. Know.=false, Time=12h, GPU=A8002026.03 | 62.1 | 36.8 | 33.3 | 43.6 | |
| Gome o3 (12h)Framework=Gome, w/o Ext. Know.=true, Time=12h, GPU=V1002026.03 | 59.1 | 20.2 | 22.2 | 32.5 | |
| AIRA o3Framework=AIRA, w/o Ext. Know.=true, Time=24h, GPU=H2002026.03 | 55 | 22 | 21.7 | 31.6 | |
| ML-Master GPT-5Framework=ML-Master, w/o Ext. Know.=true, Time=12h, GPU=V1002026.03 | 51.5 | 10.5 | 17.8 | 24 | |
| KompeteAI Gemini-2.5-flashFramework=KompeteAI, w/o Ext. Know.=false, Time=6h, GPU=A1002026.03 | 51.5 | — | — | — | |
| Gome DeepSeek-R1Framework=Gome, w/o Ext. Know.=true, Time=12h, GPU=V1002026.03 | 50 | 9.7 | 20 | 23.4 | |
| Neo Multi Frontier-Tier ModelsFramework=Neo, w/o Ext. Know.=false, Time=36h, GPU=A1002026.03 | 48.5 | 29.8 | 24.4 | 34.2 | |
| ML-Master DeepSeek-R1Framework=ML-Master, w/o Ext. Know.=true, Time=12h, GPU=V1002026.03 | 47 | 9.7 | 20 | 22.7 | |
| ML-Master o3Framework=ML-Master, w/o Ext. Know.=true, Time=12h, GPU=V1002026.03 | 42.4 | 12.3 | 20 | 22.7 | |
| AIDE o1-previewFramework=AIDE, w/o Ext. Know.=true, Time=24h, GPU=A10‡2026.03 | 34.3 | 8.8 | 10 | 16.9 | |
| AIDE gpt-4o-24-08Framework=AIDE, w/o Ext. Know.=true, Time=24h, GPU=A10‡2026.03 | 19 | 3.2 | 5.6 | 8.6 | |
| OpenHands gpt-4o-24-08Framework=OpenHands, w/o Ext. Know.=true, Time=24h, GPU=A10‡2026.03 | 11.5 | 2.2 | 1.9 | 5.1 | |
| MLAB gpt-4o-24-08Framework=MLAB, w/o Ext. Know.=true, Time=24h, GPU=A10‡2026.03 | 4.2 | 0 | 0 | 1.3 |