Functional Correctness on JSON
56.3U ScoreDreamCoder
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DreamCoderSteps=2562026.05 | 56.3 | 56.6 | 56.6 | 56.6 | 56.6 | |
| DreamCoderSteps=1282026.05 | 55.9 | 56.6 | 56.6 | 56.6 | 56.6 | |
| DreamSteps=642026.05 | 53.7 | 54.8 | 54.8 | 54.8 | 54.8 | |
| DreamCoderSteps=642026.05 | 53.7 | 53.3 | 53.3 | 53.3 | 53.3 | |
| DreamCoderSteps=322026.05 | 52.2 | 52.6 | 53.3 | 52.6 | 53.3 | |
| DreamSteps=2562026.05 | 51.8 | 53.7 | 53.7 | 53.7 | 53.7 | |
| DreamSteps=1282026.05 | 51.5 | 53.3 | 53.3 | 53.3 | 53.3 | |
| LLaDASteps=2562026.05 | 49.6 | 52.2 | 52.9 | 52.2 | 52.9 | |
| LLaDASteps=1282026.05 | 48.2 | 52.9 | 52.9 | 52.9 | 52.9 | |
| DiffuCoderSteps=322026.05 | 47.8 | 48.9 | 49.6 | 48.5 | 49.3 | |
| DiffuCoderSteps=642026.05 | 45.2 | 47.1 | 47.4 | 47.1 | 47.4 | |
| LLaDASteps=642026.05 | 44.5 | 49.6 | 50 | 49.6 | 50 | |
| LLaDASteps=322026.05 | 44.1 | 49.6 | 50 | 49.6 | 50 | |
| DreamSteps=322026.05 | 43.4 | 45.2 | 46 | 45.2 | 46 | |
| DiffuCoderSteps=162026.05 | 42.3 | 43.8 | 44.1 | 43.8 | 44.1 | |
| DiffuCoderSteps=1282026.05 | 42.3 | 43.4 | 43.8 | 43.4 | 43.8 | |
| DreamCoderSteps=162026.05 | 41.9 | 43.8 | 44.1 | 43.4 | 44 | |
| DiffuCoderSteps=2562026.05 | 41.9 | 43.4 | 43.4 | 43.4 | 43.4 | |
| LLaDASteps=162026.05 | 39 | 43.8 | 44.1 | 43.8 | 44.1 | |
| DreamSteps=162026.05 | 13.2 | 18 | 23.7 | 18 | 23.7 |