Mathematical Reasoning on AIME (AIME24/AIME25 Metrics)
60AIME 2024 ScorePolarQuant44
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| PolarQuant44Backbone=DeepSeek-R1-Distill-Llama-8B, Quantization=PolarQuant442025.02 | 60 | 36.67 | 48.33 | |
| Bf16Backbone=DeepSeek-R1-Distill-Llama-8B, Quantization=Bf162025.02 | 50 | 36.67 | 43.33 | |
| ZipCache-4Backbone=DeepSeek-R1-Distill-Llama-8B, Quantization=ZipCache-42025.02 | 43.33 | 43.33 | 43.33 | |
| KIVI-4Backbone=DeepSeek-R1-Distill-Llama-8B, Quantization=KIVI-42025.02 | 43.33 | 33.33 | 38.33 | |
| Bf16Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Quantization=Bf162025.02 | 36.67 | 23.33 | 30 | |
| PolarQuant44Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Quantization=PolarQuant442025.02 | 30 | 20 | 25 | |
| KIVI-4Backbone=DeepSeek-R1-Distill-Qwen-1.5B, Quantization=KIVI-42025.02 | 20 | 23.33 | 21.67 |