Commonsense Reasoning on HellaSwag n=50 (val)
56AccuracyHF FP16 (ceiling)
Evaluation Results
| Method | Links | |
|---|---|---|
| HF FP16 (ceiling)Calibration=— (no quant), Weights modified?=No2026.04 | 56 | |
| NVE MCAP MixedCalibration=12 prompts, 60s, Weights modified?=No2026.04 | 54 | |
| NVE W4A8 (uniform)Calibration=12 prompts, 60s, Weights modified?=No2026.04 | 54 | |
| NVE W4A16Calibration=— (no calib.), Weights modified?=No2026.04 | 54 | |
| AutoAWQ INT4Calibration=128 samples, offline, Weights modified?=Yes2026.04 | 52 |