Grade School Math Reasoning on GSM8K
28.2GSM8K AccuracyFP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Model=Llama-3.2 3B-it, # Bits (W-A-KV)=16-16-162025.06 | 28.2 | |
| FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 22.59 | |
| SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 22.29 | |
| FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 18.57 | |
| QuaRotModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 18.52 | |
| FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 17.25 | |
| RTN-optModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 12.56 | |
| FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 12.38 | |
| SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 5.61 | |
| QuaRotModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 3.74 | |
| RTN-optModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 1.14 |