Question Answering on ARC-C 0-shot
55.2AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Backbone=LLaMA-3.1-8B, Quantization=None, Protocol=0-shot2026.02 | 55.2 | |
| AstroBackbone=LLaMA-3.1-8B, Quantization=W3A16g128, Protocol=0-shot2026.02 | 48 | |
| OmniQuantBackbone=LLaMA-3.1-8B, Quantization=W3A16g128, Protocol=0-shot2026.02 | 45.1 | |
| MagRBackbone=LLaMA-3.1-8B, Quantization=W3A16g128, Protocol=0-shot2026.02 | 44.7 |