Plan Generation on Robo Challenge (Offline)
100Plan AccuracyCP-SAT Formalizer
Evaluation results
| Method | Links | |
|---|---|---|
| CP-SAT FormalizerModel=Gemini-3-flash2026.05 | 100 | |
| CP-SAT FormalizerModel=GPT-5-mini2026.05 | 99.3 | |
| CP-SAT FormalizerModel=DeepSeek-V4-Flash2026.05 | 97.9 | |
| CP-SAT FormalizerModel=Qwen3.6 35B A3B2026.05 | 97.9 | |
| PDDL2.1 FormalizerModel=Qwen3.6 35B A3B2026.05 | 67.9 | |
| PlannerModel=Qwen3.6 35B A3B2026.05 | 59.3 | |
| PDDL2.1 FormalizerModel=GPT-5-mini2026.05 | 55.7 | |
| PDDL2.1 FormalizerModel=Gemini-3-flash2026.05 | 52.9 | |
| PlannerModel=GPT-5-mini2026.05 | 46.4 | |
| PlannerModel=Gemini-3-flash2026.05 | 40.7 | |
| PDDL2.1 FormalizerModel=DeepSeek-V4-Flash2026.05 | 27.9 | |
| PlannerModel=DeepSeek-V4-Flash2026.05 | 12.9 |