Sentence completion on HellaSwag (test)
82.1AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Model=LLaMA-2-13B, Quantization=None, Calibration=FP162026.04 | 82.1 | |
| COVERCALModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=COVERCAL2026.04 | 81.7 | |
| Max-ActVarModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=Max-ActVar2026.04 | 81.1 | |
| RandomModel=LLaMA-2-13B, Quantization=AWQ INT4, Calibration Samples=128, Calibration=Random2026.04 | 80.8 | |
| Coherence Boosting (GPT-3 175B)alpha=-0.762021.10 | 72.35 | |
| GPT-3 175Balpha=-12021.10 | 62.66 | |
| GPT-3 175Balpha=02021.10 | 59.18 | |
| Coherence Boosting (GPT-2 XL)alpha=-0.782021.10 | 47.66 | |
| GPT-2 XL (1.6B)alpha=-12021.10 | 42.6 | |
| GPT-2 XL (1.6B)alpha=02021.10 | 40 | |
| PGM 6 / 6 (1024)distillation=Before2025.05 | 34.27 | |
| PGM 8 / 8distillation=Before2025.05 | 33.2 | |
| PGM 6 / 6 (1024)distillation=After Distillation (SDTT)2025.05 | 32.55 | |
| Coherence Boosting (GPT-2 Small)alpha=-0.92021.10 | 31.84 | |
| PGM 8 / 8distillation=After Distillation (SDTT)2025.05 | 31.62 | |
| MDLMdistillation=Before2025.05 | 31.36 | |
| GPT-2 Small (125M)alpha=-12021.10 | 30.99 | |
| MDLMdistillation=After Distillation (SDTT)2025.05 | 30.75 | |
| GPT-2 Small (125M)alpha=02021.10 | 28.92 |