PDE solver generation on PDE Agent Bench
60HeatAutoPDE
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| AutoPDEBackbone=GPT 5.12026.06 | 60 | 59.5 | 22.2 | 35.7 | 78.9 | 53.3 | 100 | 50 | 54.5 | |
| OpenHandsBackbone=GPT 5.12026.06 | 45 | 50 | 0 | 14.3 | 68.4 | 53.3 | 71.4 | 36.4 | 40.3 | |
| AutoPDEBackbone=Claude Opus 4.62026.06 | 40 | 33.3 | 44.4 | 46.4 | 89.5 | 100 | 71.4 | 72.7 | 54.5 | |
| OpenHandsBackbone=Claude Opus 4.62026.06 | 37.5 | 0 | 5.6 | 3.6 | 0 | 100 | 42.9 | 63.6 | 25.7 | |
| mini-swe-agentBackbone=GPT 5.12026.06 | 37.5 | 38.1 | 0 | 35.7 | 84.2 | 26.7 | 85.7 | 27.3 | 38.2 | |
| mini-swe-agentBackbone=Claude Opus 4.62026.06 | 30 | 0 | 0 | 0 | 5.3 | 93.3 | 0 | 68.2 | 22 | |
| Naive LLMBackbone=Gemini 2.5 Pro2026.06 | 20 | 2.4 | 0 | 0 | 0 | 33.3 | 14.3 | 4.5 | 8.4 | |
| Naive LLMBackbone=GPT 5.12026.06 | 20 | 23.8 | 0 | 0 | 68.4 | 33.3 | 71.4 | 0 | 21.5 | |
| Naive LLMBackbone=DeepSeek V3.22026.06 | 12.5 | 2.4 | 5.6 | 0 | 0 | 20 | 42.9 | 0 | 6.8 | |
| CodePDEBackbone=GPT 5.12026.06 | 5 | 14.3 | 0 | 0 | 36.8 | 13.3 | 42.9 | 4.5 | 11 | |
| CodePDEBackbone=Claude Opus 4.62026.06 | 0 | 9.5 | 11.1 | 50 | 94.7 | 80 | 85.7 | 22.7 | 31.9 | |
| Naive LLMBackbone=Claude Opus 4.62026.06 | 0 | 0 | 66.7 | 42.9 | 26.3 | 100 | 42.9 | 4.5 | 25.1 |