Question Answering on 8 QA Benchmarks Suite (OBQA, PIQA, ARC-E, ARC-C, HellaSwag, SciQ, WebQs, RACE) (test)
38.8OBQACRaSh
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| CRaShBackbone=OPT-6.7B, Compression Category=Training-free Compression, Setting=Plug-in2025.07 | 38.8 | 78 | 70.7 | 36.3 | 53.4 | 95.3 | 26.1 | 45.2 | 55.3 | |
| ScaleOTBackbone=LLAMA-7B, Compression Category=Post-training Compression, Setting=Plug-in2025.07 | 37.4 | 79.7 | 73.2 | 42.3 | 58.1 | 95.7 | 33.7 | 45.6 | 58.2 | |
| GradOTBackbone=OPT-6.7B, Compression Category=Training-free Compression, Setting=Plug-in2025.07 | 34.8 | 78 | 70 | 34.8 | 52.4 | 95.3 | 32.7 | 45.1 | 55.4 | |
| CRaShBackbone=LLAMA-7B, Compression Category=Training-free Compression, Setting=Plug-in2025.07 | 34.6 | 80 | 71.3 | 41.8 | 58.4 | 95.1 | 29.8 | 45.6 | 57.1 | |
| OTBackbone=OPT-6.7B, Compression Category=Training-free Compression, Setting=Plug-in2025.07 | 33.8 | 77.7 | 66.8 | 33.9 | 52.1 | 91.9 | 23.9 | 44.1 | 53 | |
| GradOTBackbone=LLAMA-7B, Compression Category=Training-free Compression, Setting=Plug-in2025.07 | 33.8 | 80.5 | 72.8 | 41.7 | 57.9 | 95.4 | 38.3 | 44.8 | 58.2 | |
| ScaleOTBackbone=OPT-6.7B, Compression Category=Post-training Compression, Setting=Plug-in2025.07 | 33.2 | 78.1 | 70.1 | 35.3 | 52.2 | 95.7 | 33.9 | 45.3 | 55.5 | |
| OTBackbone=LLAMA-7B, Compression Category=Training-free Compression, Setting=Plug-in2025.07 | 33 | 78.8 | 69.6 | 39 | 57.4 | 83.5 | 27.3 | 44 | 54.1 | |
| EmulatorBackbone=LLAMA-7B, Evaluation Protocol=Fine-tuning (FT)2025.07 | 30.6 | 76.3 | 63.3 | 32.6 | 49.4 | 91.8 | 29.6 | 43.2 | 52.1 | |
| EmulatorBackbone=OPT-6.7B, Evaluation Protocol=Fine-tuning (FT)2025.07 | 28.4 | 73.6 | 64.5 | 30.9 | 46 | 93.1 | 23.4 | 41.5 | 50.2 | |
| Full ModelBackbone=LLAMA-7B, Evaluation Protocol=Zero-shot (ZS)2025.07 | 28.2 | 78.3 | 67.3 | 38.2 | 56.4 | 89.7 | 0 | 40 | 49.8 | |
| Full ModelBackbone=OPT-6.7B, Evaluation Protocol=Zero-shot (ZS)2025.07 | 27.6 | 76.2 | 65.5 | 30.6 | 50.5 | 90.1 | 8.8 | 38.2 | 48.4 | |
| EmulatorBackbone=LLAMA-7B, Evaluation Protocol=Zero-shot (ZS)2025.07 | 23 | 72.1 | 52.8 | 30 | 45.2 | 72.6 | 0 | 32.3 | 41 | |
| EmulatorBackbone=OPT-6.7B, Evaluation Protocol=Zero-shot (ZS)2025.07 | 19.6 | 68.8 | 53.6 | 24.7 | 38.1 | 83 | 2.4 | 31.6 | 40.2 |