Commonsense Reasoning on HellaSwag 10-shot
83.3ScoreFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Model=Mistral-7B, Quantization=FP162026.04 | 83.3 | |
| COVERCALModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 82.8 | |
| FP16Model=LLaMA-3-8B, Quantization=FP162026.04 | 82.1 | |
| Max-ActVarModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 82.1 | |
| StratifiedModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 82 | |
| RandomModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 81.9 | |
| Max-PPLModel=Mistral-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 81.7 | |
| COVERCALModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 81.5 | |
| Max-ActVarModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 80.8 | |
| StratifiedModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 80.7 | |
| RandomModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 80.5 | |
| Max-PPLModel=LLaMA-3-8B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 80.3 | |
| FP16Model=LLaMA-2-7B, Quantization=FP162026.04 | 78.6 | |
| COVERCALModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 78 | |
| Max-ActVarModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 77.4 | |
| StratifiedModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 77.3 | |
| RandomModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 77.1 | |
| Max-PPLModel=LLaMA-2-7B, Quantization=AWQ INT4, K (Calibration Samples)=1282026.04 | 77 | |
| UM-190kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-190k2025.11 | 62.86 | |
| UM-187kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-187k2025.11 | 62.54 | |
| UM-170kBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UM-170k2025.11 | 62.17 | |
| TuluDPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=TuluDPO2025.11 | 61.97 | |
| ORPOBase Model=OLMo-2-7B-SFT, DPO Training Dataset=ORPO2025.11 | 61.3 | |
| HelpSteerBase Model=OLMo-2-7B-SFT, DPO Training Dataset=HelpSteer2025.11 | 60.55 | |
| UltraFBBase Model=OLMo-2-7B-SFT, DPO Training Dataset=UltraFB2025.11 | 60.24 | |
| CodePrefBase Model=OLMo-2-7B-SFT, DPO Training Dataset=CodePref2025.11 | 60.24 | |
| SFTBase Model=OLMo-2-7B-SFT, DPO Training Dataset=SFT2025.11 | 60.14 |