List your company
ResearchFormerly WizWand

Commonsense Reasoning on HellaSwag (Zero-shot Accuracy)

62.67HellaSwag Zero-shot AccuracyBase-FP16
31.771639.793347.81555.8367Sep 18, 2025Nov 4, 2025Dec 22, 2025Feb 8, 2026Mar 28, 2026May 15, 2026Jul 2, 2026
Updated 3mo ago

Evaluation results

MethodLinks
Base-FP16Model=Mistral-v0.3-7B-Instruct2025.0962.67
GPTQ-INT4Model=Mistral-v0.3-7B-Instruct2025.0962.27
Base-FP16Model=Mistral-v0.3-7B2025.0961.18
FairGPTQ-INT4Model=Mistral-v0.3-7B-Instruct2025.0960.57
GPTQ-INT4Model=Mistral-v0.3-7B2025.0960.32
FairGPTQ-INT4Model=Mistral-v0.3-7B2025.0958.7
Base-FP16Model=LLaMA-3.1-8B-Instruct2025.0957.48
GPTQ-INT4Model=LLaMA-3.1-8B-Instruct2025.0957.26
Base-FP16Model=Qwen-3-8B2025.0957.1
Base-FP16Model=Qwen-2.5-7B-Instruct2025.0957.02
FairGPTQ-INT4Model=LLaMA-3.1-8B-Instruct2025.0956.97
GPTQ-INT4Model=Qwen-2.5-7B-Instruct2025.0956.55
GPTQ-INT4Model=Qwen-3-8B2025.0956.44
FairGPTQ-INT4Model=Qwen-2.5-7B-Instruct2025.0955.46
FairGPTQ-INT4Model=Qwen-3-8B2025.0953.93
Base-FP16Model=OPT-6.7B2025.0950.51
GPTQ-INT4Model=OPT-6.7B2025.0949.76
FairGPTQ-INT4Model=OPT-6.7B2025.0946.39
Full-RankEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.0635.64
DLR + CoLAEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.0635.19
ADAMWBackbone=Llama-500M, Evaluation Protocol=Zero-shot, Communication Strategy=DENSE all-reduce2026.0734.7
ADAMSBackbone=Llama-500M, Evaluation Protocol=Zero-shot, Communication Strategy=DENSE all-reduce2026.0734.57
SCAPEBackbone=Llama-500M, Evaluation Protocol=Zero-shot, Sparsity (d)=0.12026.0734.38
SCAPEBackbone=Llama-500M, Evaluation Protocol=Zero-shot, Sparsity (d)=0.012026.0733.93
CoLAEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.0632.96