Plan Generation on Robo Challenge (Online)
85.7Plan AccuracyState-Aware CP-SAT Repair
Evaluation results
| Method | Links | |
|---|---|---|
| State-Aware CP-SAT RepairModel=Gemini-3-flash2026.05 | 85.7 | |
| State-Aware CP-SAT RepairModel=Qwen3.6 35B A3B2026.05 | 85 | |
| CP-SAT FormalizerModel=Gemini-3-flash2026.05 | 83.6 | |
| State-Aware CP-SAT RepairModel=GPT-5-mini2026.05 | 83.6 | |
| State-Aware CP-SAT RepairModel=DeepSeek-V4-Flash2026.05 | 83.6 | |
| CP-SAT FormalizerModel=Qwen3.6 35B A3B2026.05 | 72.1 | |
| PlannerModel=GPT-5-mini2026.05 | 35 | |
| PlannerModel=Qwen3.6 35B A3B2026.05 | 30.7 | |
| CP-SAT FormalizerModel=GPT-5-mini2026.05 | 25 | |
| PlannerModel=Gemini-3-flash2026.05 | 17.1 | |
| PlannerModel=DeepSeek-V4-Flash2026.05 | 12.9 | |
| CP-SAT FormalizerModel=DeepSeek-V4-Flash2026.05 | 3.6 | |
| PDDL2.1 FormalizerModel=Gemini-3-flash2026.05 | 2.9 | |
| PDDL2.1 FormalizerModel=GPT-5-mini2026.05 | 0 | |
| PDDL2.1 FormalizerModel=DeepSeek-V4-Flash2026.05 | 0 | |
| PDDL2.1 FormalizerModel=Qwen3.6 35B A3B2026.05 | 0 |