Commonsense Reasoning on HellaSwag (Zero-shot Accuracy)
62.67HellaSwag Zero-shot AccuracyBase-FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| Base-FP16Model=Mistral-v0.3-7B-Instruct2025.09 | 62.67 | |
| GPTQ-INT4Model=Mistral-v0.3-7B-Instruct2025.09 | 62.27 | |
| Base-FP16Model=Mistral-v0.3-7B2025.09 | 61.18 | |
| FairGPTQ-INT4Model=Mistral-v0.3-7B-Instruct2025.09 | 60.57 | |
| GPTQ-INT4Model=Mistral-v0.3-7B2025.09 | 60.32 | |
| FairGPTQ-INT4Model=Mistral-v0.3-7B2025.09 | 58.7 | |
| Base-FP16Model=LLaMA-3.1-8B-Instruct2025.09 | 57.48 | |
| GPTQ-INT4Model=LLaMA-3.1-8B-Instruct2025.09 | 57.26 | |
| Base-FP16Model=Qwen-3-8B2025.09 | 57.1 | |
| Base-FP16Model=Qwen-2.5-7B-Instruct2025.09 | 57.02 | |
| FairGPTQ-INT4Model=LLaMA-3.1-8B-Instruct2025.09 | 56.97 | |
| GPTQ-INT4Model=Qwen-2.5-7B-Instruct2025.09 | 56.55 | |
| GPTQ-INT4Model=Qwen-3-8B2025.09 | 56.44 | |
| FairGPTQ-INT4Model=Qwen-2.5-7B-Instruct2025.09 | 55.46 | |
| FairGPTQ-INT4Model=Qwen-3-8B2025.09 | 53.93 | |
| Base-FP16Model=OPT-6.7B2025.09 | 50.51 | |
| GPTQ-INT4Model=OPT-6.7B2025.09 | 49.76 | |
| FairGPTQ-INT4Model=OPT-6.7B2025.09 | 46.39 | |
| Full-RankEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.06 | 35.64 | |
| DLR + CoLAEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.06 | 35.19 | |
| ADAMWBackbone=Llama-500M, Evaluation Protocol=Zero-shot, Communication Strategy=DENSE all-reduce2026.07 | 34.7 | |
| ADAMSBackbone=Llama-500M, Evaluation Protocol=Zero-shot, Communication Strategy=DENSE all-reduce2026.07 | 34.57 | |
| SCAPEBackbone=Llama-500M, Evaluation Protocol=Zero-shot, Sparsity (d)=0.12026.07 | 34.38 | |
| SCAPEBackbone=Llama-500M, Evaluation Protocol=Zero-shot, Sparsity (d)=0.012026.07 | 33.93 | |
| CoLAEvaluation protocol=Zero-shot, Fine-tuning stage=Alpaca-cleaned SFT, Model scale=1B2026.06 | 32.96 |