Zero-shot Language Modeling on LM Evaluation Harness
80.66WGLO-BCQ
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| LO-BCQBackbone=Llama2-70B, Bitwidth (W4A4)=4.75, Group Size=g32, Nc=162025.02 | 80.66 | 49.28 | 84.93 | 81.34 | 65.18 | 72.28 | 0.12 | |
| BF16Backbone=Llama2-70B, Bitwidth (W4A4)=162025.02 | 79.95 | 48.8 | 85.23 | 81.56 | 65.27 | 72.16 | — | |
| LO-BCQBackbone=Llama2-70B, Bitwidth (W4A4)=4.25, Group Size=g64, Nc=22025.02 | 78.77 | 49 | 82.82 | 81.45 | 64.21 | 71.25 | 0.91 | |
| LO-BCQBackbone=Llama2-70B, Bitwidth (W4A4)=4.5, Group Size=g64, Nc=82025.02 | 78.37 | 49.28 | 84.03 | 81.45 | 64.76 | 71.58 | 0.58 | |
| VSQBackbone=Llama2-70B, Bitwidth (W4A4)=4.5, Group Size=g162025.02 | 77.27 | 47.85 | 82.29 | 79.82 | 61.4 | 69.73 | 2.43 | |
| MX4Backbone=Llama2-70B, Bitwidth (W4A4)=4.5, Group Size=g162025.02 | 76.4 | 48.04 | 82.41 | 80.58 | 63.24 | 70.13 | 2.03 | |
| MXFP4Backbone=Llama2-70B, Bitwidth (W4A4)=4.25, Group Size=g322025.02 | 76.32 | 47.75 | 83.06 | 80.58 | 63.24 | 70.19 | 1.97 | |
| QuaRotBackbone=Llama2-70B, Bitwidth (W4A4)=4.13, Group Size=g1282025.02 | 76.24 | — | — | 82.43 | — | — | — | |
| BF16Backbone=GPT3-22B, Bitwidth (W4A4)=162025.02 | 70.64 | 40.67 | 76.54 | 79.16 | 57.11 | 64.82 | — | |
| LO-BCQBackbone=GPT3-22B, Bitwidth (W4A4)=4.5, Group Size=g64, Nc=82025.02 | 70.17 | 39.43 | 77.09 | 78.62 | 56.6 | 64.38 | 0.44 | |
| LO-BCQBackbone=Llama2-7B, Bitwidth (W4A4)=4.5, Group Size=g64, Nc=82025.02 | 69.77 | 42.58 | 77.43 | 77.09 | 56.51 | 64.68 | 0.97 | |
| VSQBackbone=GPT3-22B, Bitwidth (W4A4)=4.5, Group Size=g162025.02 | 69.61 | 40.57 | 65.81 | 77.2 | 54.82 | 61.6 | 3.22 | |
| BF16Backbone=Llama2-7B, Bitwidth (W4A4)=162025.02 | 69.38 | 44.4 | 79.29 | 78.07 | 57.1 | 65.65 | — | |
| LO-BCQBackbone=GPT3-22B, Bitwidth (W4A4)=4.75, Group Size=g32, Nc=162025.02 | 69.3 | 39.62 | 75.35 | 78.89 | 56.64 | 63.96 | 0.86 | |
| LO-BCQBackbone=GPT3-22B, Bitwidth (W4A4)=4.25, Group Size=g64, Nc=22025.02 | 69.14 | 40.48 | 75.41 | 78.24 | 56.06 | 63.87 | 0.95 | |
| LO-BCQBackbone=Llama2-7B, Bitwidth (W4A4)=4.25, Group Size=g64, Nc=22025.02 | 68.9 | 42.49 | 77.58 | 77.09 | 55.93 | 64.4 | 1.25 | |
| LO-BCQBackbone=Llama2-7B, Bitwidth (W4A4)=4.75, Group Size=g32, Nc=162025.02 | 68.9 | 43.73 | 77.86 | 77.86 | 56.52 | 64.97 | 0.68 | |
| MX4Backbone=GPT3-22B, Bitwidth (W4A4)=4.5, Group Size=g162025.02 | 67.96 | 39.04 | 72.26 | 77.86 | 54.77 | 62.38 | 2.44 | |
| BF16Backbone=GPT3-8B, Bitwidth (W4A4)=162025.02 | 67.88 | 41.34 | 68.32 | 78.78 | 54.16 | 62.1 | — | |
| LO-BCQBackbone=GPT3-8B, Bitwidth (W4A4)=4.5, Group Size=g64, Nc=82025.02 | 67.72 | 39.43 | 69.45 | 77.75 | 53.71 | 61.61 | 0.49 | |
| MXFP4Backbone=Llama2-7B, Bitwidth (W4A4)=4.25, Group Size=g322025.02 | 67.48 | 41.34 | 74 | 77.53 | 54.22 | 62.91 | 2.74 | |
| MXFP4Backbone=GPT3-8B, Bitwidth (W4A4)=4.25, Group Size=g322025.02 | 67.01 | 39.71 | 65.35 | 76.12 | 50.22 | 59.68 | 2.42 | |
| LO-BCQBackbone=GPT3-8B, Bitwidth (W4A4)=4.75, Group Size=g32, Nc=162025.02 | 67 | 39.62 | 69.3 | 77.37 | 53.51 | 61.36 | 0.74 | |
| VSQBackbone=GPT3-8B, Bitwidth (W4A4)=4.5, Group Size=g162025.02 | 66.93 | 40.86 | 63.91 | 76.28 | 51.38 | 59.87 | 2.23 | |
| LO-BCQBackbone=GPT3-8B, Bitwidth (W4A4)=4.25, Group Size=g64, Nc=22025.02 | 66.85 | 40.48 | 69.2 | 77.31 | 53.06 | 61.38 | 0.72 | |
| MX4Backbone=Llama2-7B, Bitwidth (W4A4)=4.5, Group Size=g162025.02 | 66.22 | 41.43 | 73.98 | 77.04 | 55.19 | 62.77 | 2.88 | |
| MX4Backbone=GPT3-8B, Bitwidth (W4A4)=4.5, Group Size=g162025.02 | 65.11 | 38.28 | 66.27 | 75.63 | 50.77 | 59.21 | 2.89 | |
| MXFP4Backbone=GPT3-22B, Bitwidth (W4A4)=4.25, Group Size=g322025.02 | 64.17 | 39.14 | 69.61 | 75.68 | 47.6 | 59.24 | 5.58 | |
| QuaRotBackbone=Llama2-7B, Bitwidth (W4A4)=4.13, Group Size=g1282025.02 | 63.77 | — | — | 76.77 | — | — | — | |
| VSQBackbone=Llama2-7B, Bitwidth (W4A4)=4.5, Group Size=g162025.02 | 55.49 | 31.39 | 65.75 | 67.3 | 43.51 | 52.69 | 12.96 |