Zero-shot Task Performance on 0-shot6 (test)
73.33Average Accuracy (0-shot6)FP16
Evaluation results
| Method | Links | |
|---|---|---|
| FP16Model=Llama-3 8B, Precision=FP162025.06 | 73.33 | |
| FP16Model=Llama-2 13B, Precision=FP162025.06 | 72.55 | |
| FlatQuantModel=Llama-2 13B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 71.42 | |
| FlatQuantModel=Llama-3 8B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 71.23 | |
| OSTQuantModel=Llama-2 13B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 69.87 | |
| FPTQuantModel=Llama-2 13B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 69.81 | |
| FP16Model=Llama-2 7B, Precision=FP162025.06 | 69.79 | |
| SpinQuantModel=Llama-2 13B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 68.56 | |
| FPTQuantModel=Llama-3 8B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 68.41 | |
| OSTQuantModel=Llama-3 8B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 68.32 | |
| FlatQuantModel=Llama-2 7B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 67.96 | |
| SpinQuantModel=Llama-3 8B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 66.98 | |
| QuaRotModel=Llama-2 13B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 66.25 | |
| FPTQuantModel=Llama-2 7B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 66.06 | |
| OSTQuantModel=Llama-2 7B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 65.88 | |
| SpinQuantModel=Llama-2 7B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 63.52 | |
| QuaRotModel=Llama-3 8B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 61.34 | |
| QuaRotModel=Llama-2 7B, Quantization=W4A4KV4, Setting=Dynamic2025.06 | 57.73 |