Code execution on Tutorial (test)
79.51Output AccuracyCEL-S2
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| CEL-S2training_stage=Stage 2, pre-training_data=Tutorial2023.05 | 79.51 | 85.59 | 95.94 | 84.24 | 89.71 | 97.29 | 87.3 | 92.02 | |
| CodeExecutorcurriculum_learning=true2023.05 | 76.42 | 80.09 | 94.49 | 76.74 | 84.7 | 95.91 | 69.15 | 80.36 | |
| Codexshots=32023.05 | 13.07 | — | — | — | — | — | — | — | |
| CEL-S3training_stage=Stage 3, pre-training_data=CodeNetMut2023.05 | 7.89 | 8.35 | 26.58 | 21.33 | 23.67 | 26.36 | 19.47 | 22.4 |