Multiple-choice QA on Multiple-Choice Suite
72.1MC AvgERNIE
Evaluation Results
| Method | Links | |
|---|---|---|
| ERNIESparsity=Full2026.03 | 72.1 | |
| ESAPModel=ERNIE, Sparsity=25%, Criterion=EAN2026.03 | 67.5 | |
| ESAPModel=ERNIE, Sparsity=25%, Criterion=REAP2026.03 | 67.2 | |
| EANModel=ERNIE, Sparsity=25%, Allocation=UNI2026.03 | 66.9 | |
| REAPModel=ERNIE, Sparsity=25%, Allocation=UNI2026.03 | 66.7 | |
| OLMoESparsity=Full2026.03 | 65.3 | |
| ESAPModel=ERNIE, Sparsity=25%, Criterion=Freq2026.03 | 64.7 | |
| ESAPModel=ERNIE, Sparsity=25%, Criterion=SEER2026.03 | 63.8 | |
| FreqModel=ERNIE, Sparsity=25%, Allocation=UNI2026.03 | 63.6 | |
| SEERModel=ERNIE, Sparsity=25%, Allocation=UNI2026.03 | 63.4 | |
| EANModel=ERNIE, Sparsity=50%, Allocation=UNI2026.03 | 58.5 | |
| ESAPModel=ERNIE, Sparsity=50%, Criterion=EAN2026.03 | 58.2 | |
| ESAPModel=OLMoE, Sparsity=25%, Criterion=REAP2026.03 | 58.1 | |
| REAPModel=OLMoE, Sparsity=25%, Allocation=UNI2026.03 | 57.9 | |
| REAPModel=ERNIE, Sparsity=50%, Allocation=UNI2026.03 | 57.5 | |
| ESAPModel=ERNIE, Sparsity=50%, Criterion=REAP2026.03 | 57.5 | |
| FreqModel=ERNIE, Sparsity=50%, Allocation=UNI2026.03 | 56.5 | |
| ESAPModel=ERNIE, Sparsity=50%, Criterion=Freq2026.03 | 55.7 | |
| EANModel=OLMoE, Sparsity=25%, Allocation=UNI2026.03 | 55.1 | |
| SEERModel=ERNIE, Sparsity=50%, Allocation=UNI2026.03 | 55.1 | |
| FreqModel=OLMoE, Sparsity=25%, Allocation=UNI2026.03 | 54.7 | |
| ESAPModel=ERNIE, Sparsity=50%, Criterion=SEER2026.03 | 54.7 | |
| SEERModel=OLMoE, Sparsity=25%, Allocation=UNI2026.03 | 54.5 | |
| ESAPModel=OLMoE, Sparsity=25%, Criterion=EAN2026.03 | 54.3 | |
| ESAPModel=OLMoE, Sparsity=25%, Criterion=SEER2026.03 | 53.9 | |
| ESAPModel=OLMoE, Sparsity=25%, Criterion=Freq2026.03 | 53.9 | |
| FullModel=ERNIE-21B, Pruning Ratio=0%, Protocol=Zero-shot2026.03 | 0.721 | |
| EANModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.687 | |
| REAPModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.685 | |
| AIMERModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.681 | |
| RandomModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.664 | |
| SEERModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.658 | |
| FrequencyModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.655 | |
| FullModel=OLMoE-7B, Pruning Ratio=0%, Protocol=Zero-shot2026.03 | 0.653 | |
| MagnitudeModel=ERNIE-21B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.648 | |
| EANModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.605 | |
| AIMERModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.6 | |
| AIMERModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.598 | |
| REAPModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.592 | |
| REAPModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.588 | |
| RandomModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.584 | |
| EANModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.577 | |
| SEERModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.564 | |
| RandomModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.562 | |
| FrequencyModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.56 | |
| FrequencyModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.557 | |
| SEERModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.543 | |
| MagnitudeModel=ERNIE-21B, Pruning Ratio=50%, Protocol=Zero-shot2026.03 | 0.503 | |
| MagnitudeModel=OLMoE-7B, Pruning Ratio=25%, Protocol=Zero-shot2026.03 | 0.478 |