Science Question Answering on ARC Challenge (Accuracy)
66.1Accuracy (ARC)No pruning
Evaluation Results
| Method | Links | |
|---|---|---|
| No pruningSparsity=0, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 66.1 | |
| Fanar-27BSize=27B, Few-shot settings=0-shot2026.03 | 65.61 | |
| Fanar-1-9b-instructSize=9B, Few-shot settings=0-shot2026.03 | 65.19 | |
| Llama-3.3-70b-InstructSize=70B, Few-shot settings=0-shot2026.03 | 63.05 | |
| Qwen3-32BSize=32B, Few-shot settings=0-shot2026.03 | 60.84 | |
| AceGPT-v2-70B-ChatSize=70B, Few-shot settings=0-shot2026.03 | 60.07 | |
| Gemma-3-27B-itSize=27B, Few-shot settings=0-shot2026.03 | 59.98 | |
| Jais-2-70B-ChatSize=70B, Few-shot settings=0-shot2026.03 | 59.3 | |
| Allam-7B-Instruct-preview-v2Size=7B, Few-shot settings=0-shot2026.03 | 58.62 | |
| AceGPT-v2-32B-ChatSize=32B, Few-shot settings=0-shot2026.03 | 53.92 | |
| Magnitude-DimSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 50.6 | |
| DIETSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 49.6 | |
| KarnakSize=40B, Few-shot settings=0-shot2026.03 | 47.35 | |
| PuDDingSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 32.1 | |
| DIETSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 32 | |
| Magnitude-DimSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 30.8 | |
| PuDDingSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 30.3 | |
| SliceGPTSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 25 | |
| SliceGPTSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 25 |