Zero-shot Evaluation on Various Tasks Average (test)
79.95Average AccuracyFP16
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| FP16Model=LLaMA-3 70B, Precision=FP162025.11 | 79.95 | — | |
| FP16Model=LLaMA-2 70B, Precision=FP162025.11 | 77.05 | — | |
| SingleQuantModel=LLaMA-3 70B, Weight Quantization=RTN, Precision=4-bit2025.11 | 76.3 | — | |
| SingleQuantModel=LLaMA-2 70B, Weight Quantization=RTN, Precision=4-bit2025.11 | 76.14 | — | |
| SpinQuantModel=LLaMA-2 70B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 76.06 | — | |
| QuaRotModel=LLaMA-2 70B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 75.68 | — | |
| SpinQuantModel=LLaMA-2 70B, Weight Quantization=RTN, Precision=4-bit2025.11 | 75.09 | — | |
| QuaRotModel=LLaMA-2 70B, Weight Quantization=RTN, Precision=4-bit2025.11 | 73.47 | — | |
| FP16Model=LLaMA-3 8B, Precision=FP162025.11 | 73.23 | — | |
| DuQuantModel=LLaMA-3 70B, Weight Quantization=RTN, Precision=4-bit2025.11 | 72.97 | — | |
| FP16Model=LLaMA-2 13B, Precision=FP162025.11 | 72.55 | — | |
| SingleQuantModel=LLaMA-2 13B, Weight Quantization=RTN, Precision=4-bit2025.11 | 71.5 | — | |
| SpinQuantModel=LLaMA-3 70B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 71.33 | — | |
| SpinQuantModel=LLaMA-2 13B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 70.93 | — | |
| OSTQuantModel=LLaMA-2 13B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 70.27 | — | |
| FP16Model=LLaMA-2 7B, Precision=FP162025.11 | 69.87 | — | |
| DuQuantModel=LLaMA-2 70B, Weight Quantization=RTN, Precision=4-bit2025.11 | 69.39 | — | |
| OSTQuantModel=LLaMA-3 8B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 69.08 | — | |
| SingleQuantModel=LLaMA-3 8B, Weight Quantization=RTN, Precision=4-bit2025.11 | 68.96 | — | |
| QuaRotModel=LLaMA-2 13B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 68.91 | — | |
| SpinQuantModel=LLaMA-3 8B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 68.7 | — | |
| SpinQuantModel=LLaMA-2 13B, Weight Quantization=RTN, Precision=4-bit2025.11 | 68.56 | — | |
| SingleQuantModel=LLaMA-2 7B, Weight Quantization=RTN, Precision=4-bit2025.11 | 67.18 | — | |
| SpinQuantModel=LLaMA-3 8B, Weight Quantization=RTN, Precision=4-bit2025.11 | 66.98 | — | |
| OSTQuantModel=LLaMA-2 7B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 66.39 | — | |
| QuaRotModel=LLaMA-2 13B, Weight Quantization=RTN, Precision=4-bit2025.11 | 66.25 | — | |
| SpinQuantModel=LLaMA-2 7B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 66.23 | — | |
| QuaRotModel=LLaMA-3 8B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 65.79 | — | |
| DuQuantModel=LLaMA-3 8B, Weight Quantization=RTN, Precision=4-bit2025.11 | 65.76 | — | |
| SpinQuantModel=LLaMA-3 70B, Weight Quantization=RTN, Precision=4-bit2025.11 | 65.66 | — | |
| QuaRotModel=LLaMA-3 70B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 65.37 | — | |
| QuaRotModel=LLaMA-2 7B, Weight Quantization=GPTQ, Precision=4-bit2025.11 | 65.01 | — | |
| DuQuantModel=LLaMA-2 13B, Weight Quantization=RTN, Precision=4-bit2025.11 | 64.98 | — | |
| SpinQuantModel=LLaMA-2 7B, Weight Quantization=RTN, Precision=4-bit2025.11 | 63.52 | — | |
| QuaRotModel=LLaMA-3 8B, Weight Quantization=RTN, Precision=4-bit2025.11 | 61.34 | — | |
| DuQuantModel=LLaMA-2 7B, Weight Quantization=RTN, Precision=4-bit2025.11 | 61.34 | — | |
| QuaRotModel=LLaMA-2 7B, Weight Quantization=RTN, Precision=4-bit2025.11 | 57.73 | — | |
| QuaRotModel=LLaMA-3 70B, Weight Quantization=RTN, Precision=4-bit2025.11 | 35.36 | — | |
| AQLMBase Model=Llama-2-7B, #Bit=2.02, Evaluation Mode=Zero-shot2025.03 | — | 56.5 | |
| AQLMBase Model=Llama-2-13B, #Bit=1.97, Evaluation Mode=Zero-shot2025.03 | — | 60.6 | |
| BiLLMBase Model=Llama-3-70B, #Bit=1.1, Evaluation Mode=Zero-shot2025.03 | — | 44.2 | |
| ClusCompBase Model=Llama-2-7B, #Bit=2, Evaluation Mode=Zero-shot2025.03 | — | 56.6 | |
| ClusCompBase Model=Llama-2-13B, #Bit=1.99, Evaluation Mode=Zero-shot2025.03 | — | 62 | |
| ClusCompBase Model=Llama-3-8B, #Bit=2.01, Evaluation Mode=Zero-shot2025.03 | — | 53.6 | |
| ClusCompBase Model=Llama-3-70B, #Bit=1.14, Evaluation Mode=Zero-shot2025.03 | — | 39.7 | |
| ClusComp+Base Model=Llama-2-7B, #Bit=2, Evaluation Mode=Zero-shot2025.03 | — | 57.8 | |
| ClusComp+Base Model=Llama-2-13B, #Bit=1.99, Evaluation Mode=Zero-shot2025.03 | — | 63.1 | |
| ClusComp+Base Model=Llama-3-8B, #Bit=2.01, Evaluation Mode=Zero-shot2025.03 | — | 57.8 | |
| ClusComp+Base Model=Llama-3-70B, #Bit=1.14, Evaluation Mode=Zero-shot2025.03 | — | 51.4 | |
| DB-LLMBase Model=Llama-3-8B, #Bit=2, Evaluation Mode=Zero-shot2025.03 | — | 51.8 | |
| Llama-2-13BBase Model=Llama-2-13B, #Bit=16, Evaluation Mode=Zero-shot2025.03 | — | 67.8 | |
| Llama-2-7BBase Model=Llama-2-7B, #Bit=16, Evaluation Mode=Zero-shot2025.03 | — | 64.8 | |
| Llama-3-70BBase Model=Llama-3-70B, #Bit=16, Evaluation Mode=Zero-shot2025.03 | — | 75.4 | |
| Llama-3-8BBase Model=Llama-3-8B, #Bit=16, Evaluation Mode=Zero-shot2025.03 | — | 68.6 | |
| PB-LLMBase Model=Llama-3-8B, #Bit=2, Evaluation Mode=Zero-shot2025.03 | — | 38.8 | |
| PB-LLMBase Model=Llama-3-70B, #Bit=1.7, Evaluation Mode=Zero-shot2025.03 | — | 44.1 | |
| QuiPBase Model=Llama-3-8B, #Bit=2, Evaluation Mode=Zero-shot2025.03 | — | 36.8 | |
| QuIPBase Model=Llama-3-70B, #Bit=2, Evaluation Mode=Zero-shot2025.03 | — | 48.7 | |
| QuiP#Base Model=Llama-2-7B, #Bit=2.02, Evaluation Mode=Zero-shot2025.03 | — | 57.5 | |
| QuiP#Base Model=Llama-2-13B, #Bit=2.01, Evaluation Mode=Zero-shot2025.03 | — | 61.5 |