Code execution on CodeNetMut (test)
48.06Output AccuracyCodeExecutor
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| CodeExecutorcurriculum_learning=true2023.05 | 48.06 | 33.38 | 58.7 | 43.48 | 49.96 | 67.81 | 45.29 | 54.31 | |
| CodeExecutor w/o CLcurriculum_learning=false2023.05 | 45.93 | 30.98 | 60.21 | 42.45 | 49.79 | 68.55 | 41.58 | 51.76 | |
| CEL-S3training_stage=Stage 32023.05 | 43.8 | 29.44 | 59.32 | 41.76 | 49.01 | 68.3 | 41.69 | 51.78 | |
| Codexshots=32023.05 | 17.45 | — | — | — | — | — | — | — |