Code Generation on EvalPlus (test)
86.7Eval+Full
Evaluation Results
| Method | Links | |
|---|---|---|
| FullModel=ERNIE-21B, Pruning Ratio=0%, Protocol=Zero-shot2026.03 | 86.7 | |
| AIMERModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 73.4 | |
| MagnitudeModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 68.7 | |
| RandomModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 66.2 | |
| FullModel=OLMoE-7B, Pruning Ratio=0%, Protocol=Zero-shot2026.03 | 34.1 | |
| EANModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 25.8 | |
| SEERModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 25.6 | |
| FrequencyModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 25.4 | |
| AIMERModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 25.4 | |
| MagnitudeModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 24.3 | |
| RandomModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 22 | |
| REAPModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 20.8 | |
| RandomModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 11.2 | |
| AIMERModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 2.6 | |
| REAPModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 2.4 | |
| REAPModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 1.4 | |
| EANModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.9 | |
| FrequencyModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.7 | |
| SEERModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.3 | |
| FrequencyModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0 | |
| SEERModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0 | |
| EANModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0 | |
| MagnitudeModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0 |