Commonsense Reasoning on HellaSwag (HellaSwag Score and AVG)
86.86HellaSwag ScoreUNA-score & binary
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| UNA-score & binaryBase Model=Mistral 7B, Alignment Strategy=UNA-score & binary2024.08 | 86.86 | — | |
| UNA-score & binaryBase Model=Llama 8B, Alignment Strategy=UNA-score & binary2024.08 | 85.4 | — | |
| UNA-binary (BCE)Base Model=Mistral 7B, Alignment Strategy=UNA-binary (BCE)2024.08 | 84.6 | — | |
| UNA-score (MSE)Base Model=Mistral 7B, Alignment Strategy=UNA-score (MSE)2024.08 | 84.52 | — | |
| DPOBase Model=Mistral 7B, Alignment Strategy=DPO2024.08 | 84.42 | — | |
| UNA-pairwiseBase Model=Mistral 7B, Alignment Strategy=UNA-pairwise2024.08 | 84.42 | — | |
| KTOBase Model=Mistral 7B, Alignment Strategy=KTO2024.08 | 84.21 | — | |
| UNA-score (MSE)Base Model=Llama 8B, Alignment Strategy=UNA-score (MSE)2024.08 | 84.14 | — | |
| Mistral 7BBase Model=Mistral 7B, Alignment Strategy=Baseline2024.08 | 83.44 | — | |
| KTOBase Model=Llama 8B, Alignment Strategy=KTO2024.08 | 83.15 | — | |
| DPOBase Model=Llama 8B, Alignment Strategy=DPO2024.08 | 83.01 | — | |
| UNA-pairwiseBase Model=Llama 8B, Alignment Strategy=UNA-pairwise2024.08 | 83.01 | — | |
| UNA-binary (BCE)Base Model=Llama 8B, Alignment Strategy=UNA-binary (BCE)2024.08 | 83 | — | |
| UNA-score & binaryBase Model=Qwen 8B, Alignment Strategy=UNA-score & binary2024.08 | 82.92 | — | |
| Llama 8BBase Model=Llama 8B, Alignment Strategy=Baseline2024.08 | 81.78 | — | |
| UNA-score & binaryBase Model=Gemma 4B, Alignment Strategy=UNA-score & binary2024.08 | 81.21 | — | |
| UNA-score (MSE)Base Model=Qwen 8B, Alignment Strategy=UNA-score (MSE)2024.08 | 81 | — | |
| HT-MNPOReward Model=Athene-RM-8B2025.09 | 80.44 | 70.99 | |
| DPO2025.09 | 80.37 | 70.68 | |
| SFT Model2025.09 | 80.3 | 70.28 | |
| HT-MNPOReward Model=ArmoRM-Llama32025.09 | 80.29 | 70.83 | |
| INPO2025.09 | 80.22 | 70.25 | |
| HT-MNPOReward Model=Skywork-Reward-V22025.09 | 80.22 | 71.8 | |
| TD-MNPO2025.09 | 80.18 | 71.08 | |
| SPPO2025.09 | 80.1 | 70.19 | |
| Moderate-GiModel=Qwen2.52026.04 | 79.9 | — | |
| DPOBase Model=Qwen 8B, Alignment Strategy=DPO2024.08 | 79.81 | — | |
| UNA-pairwiseBase Model=Qwen 8B, Alignment Strategy=UNA-pairwise2024.08 | 79.81 | — | |
| UNA-score (MSE)Base Model=Gemma 4B, Alignment Strategy=UNA-score (MSE)2024.08 | 79.71 | — | |
| Qwen 8BBase Model=Qwen 8B, Alignment Strategy=Baseline2024.08 | 79.71 | — | |
| KTOBase Model=Qwen 8B, Alignment Strategy=KTO2024.08 | 79.58 | — | |
| RandomModel=Qwen2.52026.04 | 79.4 | — | |
| UNA-binary (BCE)Base Model=Qwen 8B, Alignment Strategy=UNA-binary (BCE)2024.08 | 79.34 | — | |
| O-LoRAModel=Qwen2.52026.04 | 79.3 | — | |
| Grad. ClipModel=Qwen2.52026.04 | 79.3 | — | |
| BaselineModel=Qwen2.52026.04 | 79.2 | — | |
| EWCModel=Qwen2.52026.04 | 79.2 | — | |
| KLModel=Qwen2.52026.04 | 79.1 | — | |
| SimPO2025.09 | 78.9 | 69.6 | |
| O-LoRAModel=LLaMA-3.12026.04 | 78.3 | — | |
| RandomModel=LLaMA-3.12026.04 | 78 | — | |
| KLModel=LLaMA-3.12026.04 | 78 | — | |
| EWCModel=LLaMA-3.12026.04 | 78 | — | |
| UNA-binary (BCE)Base Model=Gemma 4B, Alignment Strategy=UNA-binary (BCE)2024.08 | 77.95 | — | |
| BaselineModel=LLaMA-3.12026.04 | 77.9 | — | |
| Moderate-GiModel=LLaMA-3.12026.04 | 77.9 | — | |
| DPOBase Model=Gemma 4B, Alignment Strategy=DPO2024.08 | 77.83 | — | |
| UNA-pairwiseBase Model=Gemma 4B, Alignment Strategy=UNA-pairwise2024.08 | 77.83 | — | |
| KTOBase Model=Gemma 4B, Alignment Strategy=KTO2024.08 | 77.56 | — | |
| Gemma 4BBase Model=Gemma 4B, Alignment Strategy=Baseline2024.08 | 77.48 | — | |
| DPModel Scale=Gemma 9B, Method=DP2026.04 | 72.1 | — | |
| Decoupled DiLoCoModel Scale=Gemma 9B, Method=DiLoCo2026.04 | 72 | — | |
| EWCModel=Qwen32026.04 | 71.1 | — | |
| BaselineModel=Qwen32026.04 | 71 | — | |
| KLModel=Qwen32026.04 | 71 | — | |
| O-LoRAModel=Qwen32026.04 | 70.7 | — | |
| RandomModel=Qwen32026.04 | 70.6 | — | |
| Moderate-GiModel=Qwen32026.04 | 70.1 | — | |
| DPModel Scale=Gemma 5B, Method=DP2026.04 | 65.8 | — | |
| Decoupled DiLoCoModel Scale=Gemma 5B, Method=DiLoCo2026.04 | 64.9 | — | |
| DPModel Scale=Gemma 2B, Method=DP2026.04 | 52.1 | — | |
| Decoupled DiLoCoModel Scale=Gemma 2B, Method=DiLoCo2026.04 | 51.4 | — |