Machine Learning Engineering on MLE-bench-30 (test)
76Percentile RankAIRA2
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AIRA2Time budget=72h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 76 | 61.1 | 58.9 | 36.7 | |
| AIRA2 (No Subagents)Time budget=72h, Number of GPUs=8, Subagents=false, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 73.7 | 60 | 53.3 | 37.8 | |
| AIRA2Time budget=24h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 71.8 | 57.8 | 50 | 32.2 | |
| MARS+Time budget=24h, Number of GPUs=22026.03 | 69.9 | 64.4 | 51.1 | 24.4 | |
| FM-Agent 2.0Time budget=24h, Number of GPUs=12026.03 | 69.6 | 61.1 | 57.8 | 36.7 | |
| AIRA2 (No Subagents)Time budget=24h, Number of GPUs=8, Subagents=false, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 68.6 | 52.2 | 47.8 | 32.2 | |
| AIRA2 (No Evo.)Time budget=72h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=false2026.03 | 65.2 | 47.1 | 43.5 | 24.7 | |
| MLEvolveTime budget=24h, Number of GPUs=12026.03 | 64.1 | 57.8 | 52.2 | 22.2 | |
| AIRA2 (No Evo.)Time budget=24h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=false2026.03 | 64 | 45.9 | 40 | 23.5 | |
| AIRA2 (1 GPU)Time budget=72h, Number of GPUs=1, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 63.5 | 51.1 | 41.1 | 24.4 | |
| MARSTime budget=24h, Number of GPUs=12026.03 | 60.4 | 54.4 | 44.4 | 18.9 | |
| AIRA2Time budget=3h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 59.9 | 38.9 | 33.3 | 20 | |
| ML-Master 2.0Time budget=24h, Number of GPUs=22026.03 | 57.6 | 52.2 | 40 | 8.9 | |
| AIRA2 (1 GPU)Time budget=24h, Number of GPUs=1, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 56.8 | 41.1 | 32.2 | 20 | |
| AIRA2 (No HCE)Time budget=24h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=false, Evolution=true2026.03 | 56.8 | 46.5 | 45.5 | 30.7 | |
| AIRA2 (No HCE)Time budget=72h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=false, Evolution=true2026.03 | 56.3 | 47.5 | 46.5 | 32.7 | |
| AIRA2 (No Evo.)Time budget=3h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=false2026.03 | 54.7 | 35.3 | 31.8 | 16.5 | |
| AIRA2 (No Subagents)Time budget=3h, Number of GPUs=8, Subagents=false, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 54.4 | 33.3 | 30 | 13.3 | |
| PiEvolveTime budget=24h, Number of GPUs=12026.03 | 54.1 | 54.4 | 50 | 27.8 | |
| AIRA2 (No HCE)Time budget=3h, Number of GPUs=8, Subagents=true, Hidden Consistent Eval (HCE)=false, Evolution=true2026.03 | 43.4 | 29.7 | 25.7 | 12.9 | |
| AIRA2 (1 GPU)Time budget=3h, Number of GPUs=1, Subagents=true, Hidden Consistent Eval (HCE)=true, Evolution=true2026.03 | 41.3 | 22.2 | 17.8 | 10 | |
| AIRA-dojoTime budget=24h, Number of GPUs=12026.03 | 39.5 | 25.8 | 20.5 | 8.8 |