Code execution on SingleLine (test)
94.03Trace AccuracyCodeExecutor
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| CodeExecutorcurriculum_learning=true2023.05 | 94.03 | — | 94.03 | 94.03 | 94.03 | 97.28 | 97.18 | 97.23 | |
| CEL-S1training_stage=Stage 1, pre-training_data=SingleLine2023.05 | 93.32 | — | 93.32 | 93.32 | 93.32 | 96.94 | 96.86 | 96.9 | |
| Codexshots=32023.05 | 36.87 | 36.87 | 36.87 | 36.87 | 36.87 | 71.87 | 69.34 | 70.58 |