Program Synthesis on C-RASP Synthesis Benchmark Suite Regular, Counting, and Context-free Languages
27Synthesis Result ScoreC-RASP
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| C-RASPLanguage=D2, Protocol=Program Minimization, Timeout=300s2026.02 | 27 | — | — | 63.9 | 90.8 | |
| C-RASPLanguage=Next(Argmax), Protocol=Program Minimization, Timeout=300s2026.02 | 27 | — | — | 153.5 | 46.9 | |
| C-RASPLanguage=AnBnCn, Protocol=Program Minimization, Timeout=300s2026.02 | 22 | — | — | 88.7 | 42.4 | |
| C-RASPLanguage=D4, Protocol=Program Minimization, Timeout=300s2026.02 | 22 | — | — | — | — | |
| C-RASPLanguage=Tomita 7, Protocol=Program Minimization, Timeout=300s2026.02 | 22 | — | — | 79.1 | 59.8 | |
| C-RASPLanguage=Dyck-1, Protocol=Program Minimization, Timeout=300s2026.02 | 21 | — | — | 27.7 | 41.7 | |
| C-RASPLanguage=PT-3, Protocol=Program Minimization, Timeout=300s2026.02 | 21 | — | — | 91.3 | 46.4 | |
| C-RASPLanguage=Tomita 2, Protocol=Program Minimization, Timeout=300s2026.02 | 21 | — | — | 33.6 | 58.3 | |
| C-RASPLanguage=Tomita 4, Protocol=Program Minimization, Timeout=300s2026.02 | 21 | — | — | 50.2 | 52 | |
| C-RASPLanguage=D4, Protocol=Constraint Learning, Timeout=300s2026.02 | 17 | — | — | — | — | |
| C-RASPLanguage=D3, Protocol=Constraint Learning, Timeout=300s2026.02 | 15 | — | — | — | — | |
| C-RASPLanguage=Dyck-1, Protocol=Constraint Learning, Timeout=300s2026.02 | 12 | — | — | — | — | |
| C-RASPLanguage=PT-2, Protocol=Program Minimization, Timeout=300s2026.02 | 12 | — | — | 9.2 | 29.7 | |
| C-RASPLanguage=D3, Protocol=Program Minimization, Timeout=300s2026.02 | 12 | — | — | — | — | |
| C-RASPLanguage=AStarBStar, Protocol=Program Minimization, Timeout=300s2026.02 | 11 | — | — | 7.7 | 46.8 | |
| C-RASPLanguage=D2, Protocol=Constraint Learning, Timeout=300s2026.02 | 11 | — | — | 68.5 | 48.2 | |
| C-RASPLanguage=Majority, Protocol=Program Minimization, Timeout=300s2026.02 | 9 | — | — | 4.8 | 19.9 | |
| C-RASPLanguage=Tomita 2, Protocol=Constraint Learning, Timeout=300s2026.02 | 9 | — | — | 78.4 | 38.3 | |
| C-RASPLanguage=ContainsAB, Protocol=Program Minimization, Timeout=300s2026.02 | 6 | — | — | 2.2 | 12.5 | |
| C-RASPLanguage=Existential, Protocol=Program Minimization, Timeout=300s2026.02 | 6 | — | — | 2.2 | 12.5 | |
| C-RASPLanguage=Tomita 1, Protocol=Program Minimization, Timeout=300s2026.02 | 5 | — | — | 1.4 | 11.2 | |
| C-RASPLanguage=AAStar, Protocol=Program Minimization, Timeout=300s2026.02 | 4 | — | — | 1 | 10 | |
| C-RASPLanguage=Existential, Protocol=Constraint Learning, Timeout=300s2026.02 | 4 | — | — | 48 | 13.6 | |
| C-RASPLanguage=PT-3, Protocol=Constraint Learning, Timeout=300s2026.02 | 3 | — | — | — | — | |
| C-RASPLanguage=AnBnCn, Protocol=Constraint Learning, Timeout=300s2026.02 | 2 | — | — | — | — | |
| C-RASPLanguage=Tomita 4, Protocol=Constraint Learning, Timeout=300s2026.02 | 2 | — | — | — | — | |
| C-RASPLanguage=Next(Argmax), Protocol=Constraint Learning, Timeout=300s2026.02 | 2 | — | — | 185.7 | 6.3 | |
| C-RASPLanguage=AStarBStar, Protocol=Constraint Learning, Timeout=300s2026.02 | 1 | — | — | 10.2 | 8.9 | |
| C-RASPLanguage=AAStar, Protocol=Constraint Learning, Timeout=300s2026.02 | 1 | — | — | 15.5 | 8.3 | |
| C-RASPLanguage=ContainsAB, Protocol=Constraint Learning, Timeout=300s2026.02 | 1 | — | — | 145 | 10.4 | |
| C-RASPLanguage=Majority, Protocol=Constraint Learning, Timeout=300s2026.02 | 1 | — | — | 61.4 | 8.9 | |
| C-RASPLanguage=PT-2, Protocol=Constraint Learning, Timeout=300s2026.02 | 1 | — | — | 133.7 | 12 | |
| C-RASPLanguage=Tomita 1, Protocol=Constraint Learning, Timeout=300s2026.02 | 1 | — | — | 28.9 | 8.3 | |
| C-RASPLanguage=Tomita 7, Protocol=Constraint Learning, Timeout=300s2026.02 | 1 | — | — | 279.4 | 18.3 | |
| C-RASPLanguage=Dyck-1, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 3.92 | — | — | |
| C-RASPLanguage=AStarBStar, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 24.62 | — | — | |
| C-RASPLanguage=AnBnCn, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 80.05 | — | — | |
| C-RASPLanguage=AAStar, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 1.57 | — | — | |
| C-RASPLanguage=ContainsAB, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 17.35 | — | — | |
| C-RASPLanguage=Majority, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 6.02 | — | — | |
| C-RASPLanguage=Existential, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 4.62 | — | — | |
| C-RASPLanguage=PT-2, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 13.3 | — | — | |
| C-RASPLanguage=PT-3, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 59.06 | — | — | |
| C-RASPLanguage=D2, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 2.26 | — | — | |
| C-RASPLanguage=D3, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 1.54 | — | — | |
| C-RASPLanguage=D4, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 1.32 | — | — | |
| C-RASPLanguage=Tomita 1, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 3.45 | — | — | |
| C-RASPLanguage=Tomita 2, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 3.03 | — | — | |
| C-RASPLanguage=Tomita 4, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 21.32 | — | — | |
| C-RASPLanguage=Tomita 7, Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 9.32 | — | — | |
| C-RASPLanguage=Next(Argmax), Protocol=SL Synthesis, Timeout=300s2026.02 | — | 1 | 7.2 | — | — |