List your company
ResearchFormerly WizWand

Question Answering on ARC Challenge 25-shot

61.2AccuracyFP16
50.59253.34656.158.854Apr 27, 2026
Updated 6mo ago

Evaluation results

MethodLinks
FP16Model=Mistral-7B, Quantization=FP162026.0461.2
COVERCALModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0460.7
Max-ActVarModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0459.8
StratifiedModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0459.7
RandomModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0459.5
FP16Model=LLaMA-3-8B, Quantization=FP162026.0459.4
Max-PPLModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0459.3
COVERCALModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0458.7
Max-ActVarModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0457.6
StratifiedModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0457.4
RandomModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0457.2
Max-PPLModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0457
FP16Model=LLaMA-2-7B, Quantization=FP162026.0453.1
COVERCALModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0452.5
Max-ActVarModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0451.6
StratifiedModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0451.5
RandomModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0451.3
Max-PPLModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.0451