List your company
ResearchFormerly WizWand

Question Answering on AI2 Reasoning Challenge Easy (ARC-e)

83.59ARC-e AccuracyBase-FP16
62.051667.643373.23578.8267Sep 18, 2025
Updated 3mo ago

Evaluation results

MethodLinks
Base-FP16Model=Qwen-3-8B, Evaluation Protocol=Zero-shot2025.0983.59
GPTQ-INT4Model=Qwen-3-8B, Evaluation Protocol=Zero-shot2025.0981.94
Base-FP16Model=LLaMA-3.1-8B-Instruct, Evaluation Protocol=Zero-shot2025.0981.36
FairGPTQ-INT4Model=Qwen-3-8B, Evaluation Protocol=Zero-shot2025.0981.27
FairGPTQ-INT4Model=LLaMA-3.1-8B-Instruct, Evaluation Protocol=Zero-shot2025.0981.1
Base-FP16Model=Mistral-v0.3-7B, Evaluation Protocol=Zero-shot2025.0980.05
GPTQ-INT4Model=LLaMA-3.1-8B-Instruct, Evaluation Protocol=Zero-shot2025.0979.5
Base-FP16Model=Mistral-v0.3-7B-Instruct, Evaluation Protocol=Zero-shot2025.0979.38
GPTQ-INT4Model=Mistral-v0.3-7B, Evaluation Protocol=Zero-shot2025.0979.08
GPTQ-INT4Model=Mistral-v0.3-7B-Instruct, Evaluation Protocol=Zero-shot2025.0978.49
FairGPTQ-INT4Model=Mistral-v0.3-7B, Evaluation Protocol=Zero-shot2025.0976.18
FairGPTQ-INT4Model=Mistral-v0.3-7B-Instruct, Evaluation Protocol=Zero-shot2025.0975.13
FairGPTQ-INT4Model=Qwen-2.5-7B-Instruct, Evaluation Protocol=Zero-shot2025.0969.57
GPTQ-INT4Model=Qwen-2.5-7B-Instruct, Evaluation Protocol=Zero-shot2025.0969.19
Base-FP16Model=Qwen-2.5-7B-Instruct, Evaluation Protocol=Zero-shot2025.0968.52
Base-FP16Model=OPT-6.7B, Evaluation Protocol=Zero-shot2025.0966.12
GPTQ-INT4Model=OPT-6.7B, Evaluation Protocol=Zero-shot2025.0964.94
FairGPTQ-INT4Model=OPT-6.7B, Evaluation Protocol=Zero-shot2025.0962.88