Question Answering on AI2 Reasoning Challenge Easy (ARC-e)
83.59ARC-e AccuracyBase-FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| Base-FP16Model=Qwen-3-8B, Evaluation Protocol=Zero-shot2025.09 | 83.59 | |
| GPTQ-INT4Model=Qwen-3-8B, Evaluation Protocol=Zero-shot2025.09 | 81.94 | |
| Base-FP16Model=LLaMA-3.1-8B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 81.36 | |
| FairGPTQ-INT4Model=Qwen-3-8B, Evaluation Protocol=Zero-shot2025.09 | 81.27 | |
| FairGPTQ-INT4Model=LLaMA-3.1-8B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 81.1 | |
| Base-FP16Model=Mistral-v0.3-7B, Evaluation Protocol=Zero-shot2025.09 | 80.05 | |
| GPTQ-INT4Model=LLaMA-3.1-8B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 79.5 | |
| Base-FP16Model=Mistral-v0.3-7B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 79.38 | |
| GPTQ-INT4Model=Mistral-v0.3-7B, Evaluation Protocol=Zero-shot2025.09 | 79.08 | |
| GPTQ-INT4Model=Mistral-v0.3-7B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 78.49 | |
| FairGPTQ-INT4Model=Mistral-v0.3-7B, Evaluation Protocol=Zero-shot2025.09 | 76.18 | |
| FairGPTQ-INT4Model=Mistral-v0.3-7B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 75.13 | |
| FairGPTQ-INT4Model=Qwen-2.5-7B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 69.57 | |
| GPTQ-INT4Model=Qwen-2.5-7B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 69.19 | |
| Base-FP16Model=Qwen-2.5-7B-Instruct, Evaluation Protocol=Zero-shot2025.09 | 68.52 | |
| Base-FP16Model=OPT-6.7B, Evaluation Protocol=Zero-shot2025.09 | 66.12 | |
| GPTQ-INT4Model=OPT-6.7B, Evaluation Protocol=Zero-shot2025.09 | 64.94 | |
| FairGPTQ-INT4Model=OPT-6.7B, Evaluation Protocol=Zero-shot2025.09 | 62.88 |