Machine Learning Engineering on MLE-Bench 51 tasks (held-out)
58.5Avg@3MLE-IDEATOR
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| MLE-IDEATORImplementer=Claude Sonnet 3.5, IDEATOR=Sonnet 3.52026.01 | 58.5 | 60.9 | |
| MLE-IDEATOR-RLImplementer=Claude Sonnet 3.5, IDEATOR=Qwen3-8B-RL2026.01 | 58.4 | 63.1 | |
| MLE-IDEATORImplementer=Claude Sonnet 3.5, IDEATOR=Qwen3-8B2026.01 | 53.2 | 56.6 | |
| AIDEImplementer=Sonnet 3.52026.01 | 50.7 | 53.8 | |
| CodeActImplementer=Claude Sonnet 3.52026.01 | 50.6 | 52.8 | |
| AIDEImplementer=GPT-4o2026.01 | 49.6 | 50.7 | |
| CodeActImplementer=GPT-4o2026.01 | 47.9 | 51.7 | |
| MLE-IDEATOR-RLImplementer=Qwen3-8B, IDEATOR=Qwen3-8B-RL2026.01 | 29.8 | 30.1 | |
| MLE-IDEATORImplementer=Qwen3-8B, IDEATOR=Sonnet 3.52026.01 | 28 | 28.3 | |
| CodeActImplementer=Qwen3-8B2026.01 | 25.4 | 25.9 | |
| MLE-IDEATORImplementer=Qwen3-8B, IDEATOR=Qwen3-8B2026.01 | 25.2 | 25.6 |