Functional Correctness on C++
43.9Success Rate UDiffuCoder
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DiffuCoderSteps=2562026.05 | 43.9 | 44.3 | 45.5 | 44.5 | 45.7 | |
| DreamCoderSteps=1282026.05 | 28 | 28.7 | 28.7 | 28.7 | 28.7 | |
| DreamCoderSteps=2562026.05 | 28 | 27.4 | 27.4 | 27.4 | 27.4 | |
| DreamCoderSteps=642026.05 | 26.8 | 26.8 | 27.2 | 26.2 | 26.6 | |
| LLaDASteps=2562026.05 | 25.6 | 25 | 25 | 25 | 25 | |
| DiffuCoderSteps=1282026.05 | 25.6 | 27.4 | 29.9 | 27.4 | 29.7 | |
| DreamCoderSteps=322026.05 | 19.5 | 20.1 | 20.7 | 20.7 | 21.3 | |
| DreamSteps=642026.05 | 18.1 | 18.5 | 18.5 | 18.9 | 18.9 | |
| DiffuCoderSteps=642026.05 | 17.7 | 18.3 | 18.3 | 18.3 | 18.3 | |
| LLaDASteps=1282026.05 | 15.9 | 15.9 | 15.9 | 15.9 | 15.9 | |
| DreamSteps=322026.05 | 15.2 | 16.5 | 16.5 | 16.5 | 16.5 | |
| DiffuCoderSteps=322026.05 | 15.2 | 18.3 | 18.3 | 17.7 | 17.7 | |
| DreamSteps=1282026.05 | 13.8 | 14.6 | 14.6 | 14.4 | 14.4 | |
| DreamSteps=2562026.05 | 12.8 | 13.4 | 13.4 | 14 | 14 | |
| DiffuCoderSteps=162026.05 | 12.8 | 14 | 14 | 14.6 | 14.6 | |
| DreamSteps=162026.05 | 10.4 | 11.6 | 11.6 | 11.6 | 11.6 | |
| LLaDASteps=642026.05 | 10.4 | 11 | 11 | 11 | 11 | |
| DreamCoderSteps=162026.05 | 6.7 | 7.9 | 8.5 | 9.1 | 9.1 | |
| LLaDASteps=322026.05 | 3 | 5.1 | 5.3 | 5.5 | 6.3 | |
| LLaDASteps=162026.05 | 0.6 | 2.8 | 4.1 | 3.3 | 4.5 |