Commonsense Reasoning on HellaSwag
99.21AccuracyCOLLATE
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| COLLATEBackbone=open-llama-v2-7b2025.06 | 99.21 | — | — | — | — | — | — | — | — | — | — | |
| SPINCategory=Training-driven Rationale Enhancement, Backbone=open-llama-v2-7b2025.06 | 98.14 | — | — | — | — | — | — | — | — | — | — | |
| Flexora (w/ LoReFT)Backbone Model=Llama3-8B, LoRA Algorithm=LoReFT, Flexora Integration=true2024.08 | 96.47 | — | — | — | — | — | — | — | — | — | — | |
| LoReFTBackbone Model=Llama3-8B, LoRA Algorithm=LoReFT, Flexora Integration=false2024.08 | 96.31 | — | — | — | — | — | — | — | — | — | — | |
| Dense ModelBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 96.3 | — | — | — | — | — | — | — | — | — | — | |
| GradPrunerBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 96.3 | — | — | — | — | — | — | — | — | — | — | |
| Distilling Step-by-StepCategory=Task-specific Supervised Fine-Tuning, Backbone=open-llama-v2-7b2025.06 | 96.01 | — | — | — | — | — | — | — | — | — | — | |
| Dense ModelBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 95.9 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Gemma-7B2024.08 | 95.76 | — | — | — | — | — | — | — | — | — | — | |
| Hard-routing MoEBackbone=Qwen2.5-7B-Instruct, # Params (%)=14.2G (100%), # Trans. (%)=14.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 95.58 | — | — | — | — | — | — | — | — | — | — | |
| GradPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 95.4 | — | — | — | — | — | — | — | — | — | — | |
| GPT-4Number of shots=10-shot2023.11 | 95.3 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Mistral-7B-v0.12024.08 | 95.08 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Zephyr-7B-beta2024.08 | 95.05 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Gemma-7B2024.08 | 94.85 | — | — | — | — | — | — | — | — | — | — | |
| Flexora (w/ rsLoRA)Backbone Model=Llama3-8B, LoRA Algorithm=rsLoRA, Flexora Integration=true2024.08 | 94.83 | — | — | — | — | — | — | — | — | — | — | |
| open-llama-7b-v2 SFTCategory=Task-specific Supervised Fine-Tuning, Backbone=open-llama-v2-7b2025.06 | 94.79 | — | — | — | — | — | — | — | — | — | — | |
| Kimi-K2 Base# Shots=10-shot, # Activated Params=32B, # Total Params=1043B2026.01 | 94.6 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Mistral-7B-v0.12024.08 | 94.35 | — | — | — | — | — | — | — | — | — | — | |
| rsLoRABackbone Model=Llama3-8B, LoRA Algorithm=rsLoRA, Flexora Integration=false2024.08 | 94.33 | — | — | — | — | — | — | — | — | — | — | |
| Dense ModelBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=0%2026.01 | 94.3 | — | — | — | — | — | — | — | — | — | — | |
| Flexora (w/ DoRA)Backbone Model=Llama3-8B, LoRA Algorithm=DoRA, Flexora Integration=true2024.08 | 94.1 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Qwen2.5-32B2024.08 | 94.01 | — | — | — | — | — | — | — | — | — | — | |
| Supervised SOTAsupervised=true2022.03 | 93.9 | — | — | — | — | — | — | — | — | — | — | |
| GradPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 93.9 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Zephyr-7B-beta2024.08 | 93.77 | — | — | — | — | — | — | — | — | — | — | |
| Flexora (w/ MoSLoRA)Backbone Model=Llama3-8B, LoRA Algorithm=MoSLoRA, Flexora Integration=true2024.08 | 93.76 | — | — | — | — | — | — | — | — | — | — | |
| Flexora (w/ LoRA)Backbone Model=Llama3-8B, LoRA Algorithm=LoRA, Flexora Integration=true2024.08 | 93.62 | — | — | — | — | — | — | — | — | — | — | |
| DoRABackbone Model=Llama3-8B, LoRA Algorithm=DoRA, Flexora Integration=false2024.08 | 93.62 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Llama3-8B2024.08 | 93.62 | — | — | — | — | — | — | — | — | — | — | |
| MoSLoRABackbone Model=Llama3-8B, LoRA Algorithm=MoSLoRA, Flexora Integration=false2024.08 | 93.53 | — | — | — | — | — | — | — | — | — | — | |
| Self-Rewarding LMsCategory=Training-driven Rationale Enhancement, Backbone=open-llama-v2-7b2025.06 | 93.26 | — | — | — | — | — | — | — | — | — | — | |
| FT(Llama3.2)Base Model=Llama3.2-3B, Fine-tuning Protocol=FFT, Sparsity=0%2026.01 | 93.2 | — | — | — | — | — | — | — | — | — | — | |
| SATBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 93.1 | — | — | — | — | — | — | — | — | — | — | |
| FT(Llama3.2)Base Model=Llama3.2-3B, Fine-tuning Protocol=LoRA, Sparsity=0%2026.01 | 92.6 | — | — | — | — | — | — | — | — | — | — | |
| APTBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 92.6 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Yi-6B2024.08 | 92.24 | — | — | — | — | — | — | — | — | — | — | |
| APTBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 92.2 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Qwen1.5-7B2024.08 | 91.96 | — | — | — | — | — | — | — | — | — | — | |
| FPFTBackbone=Qwen2.5-7B-Instruct, # Params (%)=14.2G (100%), # Trans. (%)=14.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 91.8 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Qwen1.5-7B2024.08 | 91.75 | — | — | — | — | — | — | — | — | — | — | |
| Sparse-and-Orthogonal LoRABackbone=Qwen2.5-7B-Instruct, # Params (%)=87M (0.60%), # Trans. (%)=42M (0.30%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 91.32 | — | — | — | — | — | — | — | — | — | — | |
| Hard-routing MoEBackbone=Qwen2.5-1.5B-Instruct, # Params (%)=3.2G (100%), # Trans. (%)=3.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 90.91 | — | — | — | — | — | — | — | — | — | — | |
| SATBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 90.9 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Llama2-7B2024.08 | 90.89 | — | — | — | — | — | — | — | — | — | — | |
| SATBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 90.8 | — | — | — | — | — | — | — | — | — | — | |
| LacoBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 90.7 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Vicuna-7B-v1.52024.08 | 90.43 | — | — | — | — | — | — | — | — | — | — | |
| LLMPrunerBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 90.4 | — | — | — | — | — | — | — | — | — | — | |
| LacoBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 90.4 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Qwen2.5-32B2024.08 | 90.35 | — | — | — | — | — | — | — | — | — | — | |
| LacoBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 90.1 | — | — | — | — | — | — | — | — | — | — | |
| MINITRONBase Model=Mistral-7B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 90 | — | — | — | — | — | — | — | — | — | — | |
| MINITRONBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 89.9 | — | — | — | — | — | — | — | — | — | — | |
| LLMPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 89.8 | — | — | — | — | — | — | — | — | — | — | |
| MINITRONBase Model=Llama3.1-8B, Fine-tuning Protocol=FFT, Sparsity=40%2026.01 | 89.8 | — | — | — | — | — | — | — | — | — | — | |
| Sparse-and-Orthogonal LoRA (Single)Backbone=Qwen2.5-7B-Instruct, # Params (%)=87M (0.60%), # Trans. (%)=42M (0.30%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 89.78 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone Model=Llama3-8B, LoRA Algorithm=LoRA, Flexora Integration=false2024.08 | 89.72 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Llama3-8B2024.08 | 89.72 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Yi-6B2024.08 | 89.46 | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.2 Exp Base# Shots=10-shot, # Activated Params=37B, # Total Params=671B2026.01 | 89.4 | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.1 Base# Shots=10-shot, # Activated Params=37B, # Total Params=671B2026.01 | 89.2 | — | — | — | — | — | — | — | — | — | — | |
| LoRIBackbone=Qwen2.5-7B-Instruct, # Params (%)=168M (1.16%), # Trans. (%)=168M (1.16%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 89.12 | — | — | — | — | — | — | — | — | — | — | |
| Falcon-180BNumber of shots=10-shot2023.11 | 89 | — | — | — | — | — | — | — | — | — | — | |
| FalconModel Size=180B, Evaluation Protocol=10-shot2023.11 | 89 | — | — | — | — | — | — | — | — | — | — | |
| Mixtral-8x22B2024.07 | 88.7 | — | — | — | — | — | — | — | — | — | — | |
| MiMo-V2-Flash Base# Shots=10-shot, # Activated Params=15B, # Total Params=309B2026.01 | 88.5 | — | — | — | — | — | — | — | — | — | — | |
| FlexoraBackbone=Xuan Yuan-6B2024.08 | 88.41 | — | — | — | — | — | — | — | — | — | — | |
| LLMPrunerBase Model=Llama3.1-8B, Fine-tuning Protocol=LoRA, Sparsity=40%2026.01 | 88.3 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Qwen2.5-7B-Instruct, # Params (%)=362M (2.5%), # Trans. (%)=362M (2.5%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 88.18 | — | — | — | — | — | — | — | — | — | — | |
| Llama-3-70B2024.07 | 88 | — | — | — | — | — | — | — | — | — | — | |
| FP16Backbone=Qwen2.5-7B, Quantization=FP162026.01 | 87.79 | — | — | — | — | — | — | — | — | — | — | |
| CLGModel=Qwen2.5-72B, Shots=1282025.06 | 87.68 | — | — | — | — | — | — | — | — | — | — | |
| CLGShot count=128-shot, Model=Qwen2.5-72B2025.06 | 87.68 | — | — | — | — | — | — | — | — | — | — | |
| LoRABackbone=Vicuna-7B-v1.52024.08 | 87.64 | — | — | — | — | — | — | — | — | — | — | |
| Qwen2-72B2024.07 | 87.6 | — | — | — | — | — | — | — | — | — | — | |
| Qwen1.5-110B2024.07 | 87.5 | — | — | — | — | — | — | — | — | — | — | |
| DeepSeek-R1 (BF16)Model=DeepSeek-R1, Setting=W16A16, Sparsity=0, Evaluation Protocol=Non-generative2025.12 | 87.43 | — | — | — | — | — | — | — | — | — | — | |
| FPFTBackbone=Qwen2.5-1.5B-Instruct, # Params (%)=3.2G (100%), # Trans. (%)=3.2G (100%), Fine-tuning setting=DFL, Rank (rl)=322026.02 | 87.35 | — | — | — | — | — | — | — | — | — | — | |
| LLaMA-2Model Size=70B, Evaluation Protocol=10-shot2023.11 | 87.3 | — | — | — | — | — | — | — | — | — | — | |
| Mistral Small 24B Inst 2501Classifier=Self-labeled2026.01 | 87.2 | 0.9 | — | — | — | — | — | — | — | — | — | |
| SQ-formatModel=DeepSeek-R1, Setting=W(SQ5)A8, Sparsity=0.875, Evaluation Protocol=Non-generative2025.12 | 87.19 | — | — | — | — | — | — | — | — | — | — | |
| LLaMA 2 70B10-shot=true2024.01 | 87.1 | — | — | — | — | — | — | — | — | — | — | |
| JambaArchitecture=MoE, # Act Params=12B, # Params=52B2024.07 | 87.1 | — | — | — | — | — | — | — | — | — | — | |
| HICDBase Model=Mistral-7B-v0.32025.03 | 87.1 | — | — | — | — | — | — | — | — | — | — | |
| Llama-2 (70B)Implementation=ANN, Bit-channels (Ch.)=322026.01 | 86.9 | — | — | — | — | — | — | — | — | — | — | |
| Llama-2 (70B)Implementation=SNN (NEXUS), Bit-channels (Ch.)=322026.01 | 86.9 | — | — | — | — | — | — | — | — | — | — | |
| Best-of-NModel=Qwen2.5-72B, Shots=1282025.06 | 86.83 | — | — | — | — | — | — | — | — | — | — | |
| PaLM 2-Lprompting=1-shot2023.05 | 86.8 | — | — | — | — | — | — | — | — | — | — | |
| Latent-BayesianModel=Qwen2.5-72B, Shots=1282025.06 | 86.72 | — | — | — | — | — | — | — | — | — | — | |
| Mixtral 8x7B10-shot=true2024.01 | 86.7 | — | — | — | — | — | — | — | — | — | — | |
| BGE-KMeansModel=Qwen2.5-72B, Shots=1282025.06 | 86.67 | — | — | — | — | — | — | — | — | — | — | |
| HiF4Backbone=DeepSeek-V3.1 671B2026.02 | 86.6 | — | — | — | — | — | 2.19 | — | — | — | — | |
| RandomModel=Qwen2.5-72B, Shots=1282025.06 | 86.58 | — | — | — | — | — | — | — | — | — | — | |
| RandomShot count=128-shot, Model=Qwen2.5-72B2025.06 | 86.58 | — | — | — | — | — | — | — | — | — | — | |
| Mixtral-8x7BArchitecture=MoE, # Act Params=12B, # Params=47B2024.07 | 86.5 | — | — | — | — | — | — | — | — | — | — | |
| Mixtral 8x7B Inst.Shots=10-shot2024.01 | 86.5 | — | — | — | — | — | — | — | — | — | — | |
| RoBERTa-L + MUPPETModel Size=Large, Backbone=RoBERTa, Training Strategy=MUPPET2021.01 | 86.4 | — | — | — | — | — | — | — | — | — | — | |
| HICDBase Model=LLaMA3-8B-Instruct2025.03 | 86.4 | — | — | — | — | — | — | — | — | — | — | |
| EPR-KMeansModel=Qwen2.5-72B, Shots=1282025.06 | 86.22 | — | — | — | — | — | — | — | — | — | — |