Zero-shot Evaluation on Benchmarks 0-shot
73.330-shot Average ScoreFP16
Evaluation Results
| Method | Links | |
|---|---|---|
| FP16Model backbone=Llama-3 8B2025.06 | 73.33 | |
| FP16Model backbone=Llama-2 13B2025.06 | 72.55 | |
| FlatQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 71.64 | |
| FlatQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 71.42 | |
| FlatQuantWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 71.33 | |
| FlatQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 71.23 | |
| SpinQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 70.93 | |
| OSTQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 70.03 | |
| FPTQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 69.97 | |
| OSTQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 69.87 | |
| FPTQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 69.81 | |
| FP16Model backbone=Llama-2 7B2025.06 | 69.79 | |
| QuaRotWeight Quantizer=GPTQ, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 68.91 | |
| SpinQuantWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 68.7 | |
| FPTQuantWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 68.7 | |
| OSTQuantWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 68.64 | |
| SpinQuantWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 68.56 | |
| FPTQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 68.41 | |
| OSTQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 68.32 | |
| FlatQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 67.96 | |
| FlatQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 67.47 | |
| SpinQuantWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 66.98 | |
| OSTQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 66.58 | |
| FPTQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 66.44 | |
| QuaRotWeight Quantizer=RTN, Model backbone=Llama-2 13B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 66.25 | |
| SpinQuantWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 66.23 | |
| FPTQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 66.06 | |
| OSTQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 65.88 | |
| QuaRotWeight Quantizer=GPTQ, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 65.79 | |
| QuaRotWeight Quantizer=GPTQ, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 65.01 | |
| SpinQuantWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 63.52 | |
| QuaRotWeight Quantizer=RTN, Model backbone=Llama-3 8B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 61.34 | |
| QuaRotWeight Quantizer=RTN, Model backbone=Llama-2 7B, Quantization format=W4A4KV4, Quantization mode=Dynamic2025.06 | 57.73 |