Commonsense Reasoning on HellaSwag (Accuracy)
87.4HellaSwag AccuracyMSRS
Evaluation results
| Method | Links | |
|---|---|---|
| MSRSBackbone=Mistral-7B-v0.32025.08 | 87.4 | |
| LoPT-GRPOBackbone=Qwen2.5-32B2026.05 | 86.24 | |
| VanillaBackbone=Mistral-7B-v0.32025.08 | 86.2 | |
| BaseBackbone=Qwen2.5-32B2026.05 | 85.9 | |
| E2E-GRPOBackbone=Qwen2.5-32B2026.05 | 85.87 | |
| LLaMA-2 70BModel=LLaMA-2 70B2026.03 | 84 | |
| Focus (LLaMA-2 70B)Backbone=LLaMA-2 70B, K=4, dg=162026.03 | 84 | |
| MSRSBackbone=Llama-3-8B-Instruct2025.08 | 83.9 | |
| MSRSBackbone=Qwen2-7B-Instruct2025.08 | 83.5 | |
| VanillaBackbone=Qwen2-7B-Instruct2025.08 | 83.1 | |
| BaselineModel=Qwen3 32B2026.03 | 82.7 | |
| Mistral 7BModel=Mistral 7B2026.03 | 81.2 | |
| Focus (Mistral 7B)Backbone=Mistral 7B, K=4, dg=162026.03 | 81.2 | |
| Mistral-7BRatio=0%, Evaluation Protocol=zero-shot2026.03 | 81.02 | |
| BaselineModel=Falcon3 10B2026.03 | 80.95 | |
| SymNoiseBackbone=Llama-2 7B, Instruction-tuning dataset=OpenPlatypus, Noise strategy=SymNoise2026.05 | 80.8 | |
| I-DPO + MaPPOBase Model=Llama-3-8B-Instruct, DPO Mode=iterative2026.05 | 80.7 | |
| BaselineModel=Qwen3-4B2026.06 | 80.61 | |
| NEFTuneBackbone=Llama-2 7B, Instruction-tuning dataset=OpenPlatypus, Noise strategy=NEFTune2026.05 | 80.6 | |
| ERSSModel=Qwen3-4B2026.06 | 80.6 | |
| LRSModel=Qwen3-4B2026.06 | 80.59 | |
| ALRRModel=Qwen3-4B2026.06 | 80.58 | |
| OLMO-2Model Type=Fully-open, Number of Parameters=7B2026.03 | 80.5 | |
| OLMo-2 7BModel=OLMo-2 7B2026.03 | 80.5 | |
| Focus (OLMo-2 7B)Backbone=OLMo-2 7B, K=4, dg=162026.03 | 80.5 | |
| I-DPOBase Model=Llama-3-8B-Instruct, DPO Mode=iterative2026.05 | 80.5 | |
| CTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 80.47 | |
| float16Model=Mistral-7B-v0.3, Precision=float16, Evaluation Protocol (Shots)=0-shot2024.12 | 80.43 | |
| MixLLMModel=Mistral-7B-v0.3, Precision=W8A8, Evaluation Protocol (Shots)=0-shot2024.12 | 80.4 | |
| Llama-2 7BBackbone=Llama-2 7B, Instruction-tuning dataset=OpenPlatypus, Noise strategy=None2026.05 | 80.4 | |
| C32A2Base Model=Qwen2.5-7B, #P/B=184.42, #AP/B=13.332026.03 | 80.38 | |
| PTBase Model=Qwen2.5-7B, #P/B=7.62, #AP/B=7.622026.03 | 80.26 | |
| SmoothQuantModel=Mistral-7B-v0.3, Precision=W8A8, Evaluation Protocol (Shots)=0-shot2024.12 | 80.24 | |
| MixLLMModel=Mistral-7B-v0.3, Precision=W4.4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 80.21 | |
| SymNoiseBackbone=Llama-2 7B, Instruction-tuning dataset=Alpaca, Noise strategy=SymNoise2026.05 | 80.2 | |
| Qwen2.5-7BModel Backbone=Qwen2.5-7B2026.06 | 80.2 | |
| Qwen2.5-7BModel=Qwen2.5-7B2026.06 | 80.2 | |
| Phi-4 14B BaseModel=Phi-4 14B, Variant=Base2026.06 | 80.2 | |
| VanillaBackbone=Llama-3-8B-Instruct2025.08 | 80.1 | |
| DPO + MaPPOBase Model=Llama-3-8B-Instruct, DPO Mode=offline2026.05 | 80.1 | |
| NEFTuneBackbone=Llama-2 7B, Instruction-tuning dataset=Alpaca, Noise strategy=NEFTune2026.05 | 80.1 | |
| Phi-4 14B Chapter IModel=Phi-4 14B, Variant=Chapter I2026.06 | 80.1 | |
| Llama-2 7BBackbone=Llama-2 7B, Instruction-tuning dataset=Alpaca, Noise strategy=None2026.05 | 80 | |
| QuaRotModel=Mistral-7B-v0.3, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 79.98 | |
| Gemma 2 9BModel=Gemma 2 9B2026.03 | 79.9 | |
| Focus (Gemma 2 9B)Backbone=Gemma 2 9B, K=4, dg=162026.03 | 79.9 | |
| QServeModel=Mistral-7B-v0.3, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 79.89 | |
| MixLLMModel=Qwen2.5-7B, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 79.71 | |
| Full modelSparsity=0%, Backbone=Qwen3-30B-A3B, Evaluation Protocol=Zero-shot2026.05 | 79.7 | |
| CleanBackbone=LLaMA2-13B-Chat2026.03 | 79.63 | |
| LLaMA-2 13BModel=LLaMA-2 13B2026.03 | 79.6 | |
| Focus (LLaMA-2 13B)Backbone=LLaMA-2 13B, K=2, dg=162026.03 | 79.6 | |
| BF16-BaselineEff. Bits=16, Model=Llama-3.1-8B-Instruct2026.06 | 79.52 | |
| FineRMoEBase Model=Qwen2.5-7B, #P/B=26.65, #AP/B=7.942026.03 | 79.51 | |
| DPOBase Model=Llama-3-8B-Instruct, DPO Mode=offline2026.05 | 79.5 | |
| Llama 3.1 8B BaseModel=Llama 3.1 8B, Variant=Base2026.06 | 79.5 | |
| float16Model=Average, Precision=float16, Evaluation Protocol (Shots)=0-shot2024.12 | 79.43 | |
| MixLLMModel=Average, Precision=W8A8, Evaluation Protocol (Shots)=0-shot2024.12 | 79.42 | |
| BaselineModel=Llama3.1 8B2026.03 | 79.28 | |
| FullBits=16, Size=30B, Evaluation Protocol=zero-shot2026.04 | 79.21 | |
| GRINQH-8b-RTNEff. Bits=3.96, Model=Llama-3.1-8B-Instruct2026.06 | 79.21 | |
| SmoothQuantModel=Average, Precision=W8A8, Evaluation Protocol (Shots)=0-shot2024.12 | 79.2 | |
| Ministral 8B BaseModel=Ministral 8B, Variant=Base2026.06 | 79.1 | |
| NVShardBase Model=Qwen2.5-7B, #P/B=47.55, #AP/B=7.632026.03 | 79.05 | |
| MixLLMModel=Average, Precision=W4.4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 79 | |
| Qwen3 14B Chapter IModel=Qwen3 14B, Variant=Chapter I2026.06 | 79 | |
| Phi-4 14B Chapter IIModel=Phi-4 14B, Variant=Chapter II2026.06 | 79 | |
| Qwen2.5 7BModel Architecture=AR, Training tokens=18T2026.06 | 79 | |
| MixLLMModel=Llama-3.1-8B, Precision=W8A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.98 | |
| GRINQH-6b-RTNEff. Bits=3.881, Model=Llama-3.1-8B-Instruct2026.06 | 78.95 | |
| float16Model=Qwen2.5-7B, Precision=float16, Evaluation Protocol (Shots)=0-shot2024.12 | 78.94 | |
| float16Model=Llama-3.1-8B, Precision=float16, Evaluation Protocol (Shots)=0-shot2024.12 | 78.92 | |
| SymNoiseBackbone=Llama-1 7B, Instruction-tuning dataset=Evol-Instruct, Noise strategy=SymNoise2026.05 | 78.9 | |
| Qwen2.5-7BType=AR2026.06 | 78.9 | |
| SmoothQuantModel=Llama-3.1-8B, Precision=W8A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.88 | |
| MixLLMModel=Qwen2.5-7B, Precision=W8A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.88 | |
| Qwen3 14B BaseModel=Qwen3 14B, Variant=Base2026.06 | 78.8 | |
| GRINQH-8b-RTNEff. Bits=2.95, Model=Llama-3.1-8B-Instruct2026.06 | 78.68 | |
| BD-VAXBackbone=LLaMA2-13B-Chat, Parameter Setting=LoRA Adapter2026.03 | 78.67 | |
| MixLLMModel=Average, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.61 | |
| ITBase Model=Llama-3-8B-Instruct2026.05 | 78.6 | |
| Qwen3-8BType=AR2026.06 | 78.6 | |
| MixLLMModel=Qwen2.5-7B, Precision=W4.4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.58 | |
| QuaRotModel=Average, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.55 | |
| BD-VAXBackbone=LLaMA2-13B-Chat, Parameter Setting=Full Params2026.03 | 78.52 | |
| Qwen3 14B Chapter IIModel=Qwen3 14B, Variant=Chapter II2026.06 | 78.5 | |
| SmoothQuantModel=Qwen2.5-7B, Precision=W8A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.48 | |
| GRINQH-4b-GPTQEff. Bits=3.01, Model=Llama-3.1-8B-Instruct2026.06 | 78.36 | |
| Qwen 2.5 7BModel=Qwen 2.5 7B2026.03 | 78.3 | |
| Focus (Qwen 2.5 7B)Backbone=Qwen 2.5 7B, K=4, dg=162026.03 | 78.3 | |
| SEPTQBits=3, Size=30B, Evaluation Protocol=zero-shot2026.04 | 78.21 | |
| MixLLMModel=Llama-3.1-8B, Precision=W4.4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.2 | |
| MixLLMModel=Mistral-7B-v0.3, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.17 | |
| QuaRotModel=Mistral-7B-v0.3, Precision=W4A4, Evaluation Protocol (Shots)=0-shot2024.12 | 78.14 | |
| QServeModel=Average, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 78.1 | |
| BF16Models=DeepSeek-V2-Lite, Quantization Strategy=Embedding-wise, Quantization Bit-width=BF162026.04 | 78 | |
| NEFTuneBackbone=Llama-1 7B, Instruction-tuning dataset=Evol-Instruct, Noise strategy=NEFTune2026.05 | 78 | |
| MixLLMModel=Llama-3.1-8B, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 77.94 | |
| Llama-1 7BBackbone=Llama-1 7B, Instruction-tuning dataset=Evol-Instruct, Noise strategy=None2026.05 | 77.9 | |
| QuaRotModel=Qwen2.5-7B, Precision=W4A8, Evaluation Protocol (Shots)=0-shot2024.12 | 77.84 |