Massive Multitask Language Understanding on MMLU (5-shot)
59.69MMLU (5-shot)FP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Model=Llama-3.2 3B-it, # Bits (W-A-KV)=16-16-162025.06 | 59.69 | |
| FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 55.46 | |
| FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 52.62 | |
| SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 52.14 | |
| QuaRotModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 51.68 | |
| FlatQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 49.01 | |
| RTN-optModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-8-42025.06 | 46.45 | |
| FPTQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 43.71 | |
| SpinQuantModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 38.17 | |
| QuaRotModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 35.72 | |
| RTN-optModel=Llama-3.2 3B-it, # Bits (W-A-KV)=4-4-42025.06 | 24.98 |