Accuracy on PiQA (Question Answering)
81.77AccuracyLO-BCQ (g128)
Evaluation Results
| Method | Links | |
|---|---|---|
| LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.31, #codebooks=42025.02 | 81.77 | |
| AQLMBackbone=Llama2-70B, Bitwidth=4.14, #codebooks=2 * 2^162025.02 | 81.5 | |
| LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.44, #codebooks=82025.02 | 81.5 | |
| QuiP#Backbone=Llama2-70B, Bitwidth=4.02, #codebooks=2 * 2^162025.02 | 81.45 | |
| LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.56, #codebooks=162025.02 | 81.45 | |
| GPTQ (g128)Backbone=Llama2-70B, Bitwidth=42025.02 | 81.23 | |
| LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.19, #codebooks=22025.02 | 81.18 | |
| QuIP#Avg. Bitwidth=4.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 78.4 | |
| QTIPAvg. Bitwidth=4.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 78.4 | |
| SSQR-1%Avg. Bitwidth=4.445, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 78.35 | |
| LO-BCQ (g128)Backbone=Llama2-7B, Bitwidth=4.19, #codebooks=22025.02 | 78.29 | |
| AQLMAvg. Bitwidth=5.020, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 78.29 | |
| AQLMBackbone=Llama2-7B, Bitwidth=4.14, #codebooks=2 * 2^162025.02 | 78.24 | |
| AQLMAvg. Bitwidth=4.040, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 78.24 | |
| LO-BCQ (g128)Backbone=Llama2-7B, Bitwidth=4.31, #codebooks=42025.02 | 78.18 | |
| FP16 BaselineAvg. Bitwidth=16.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 78.13 | |
| QTIPAvg. Bitwidth=3.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 78.1 | |
| LO-BCQ (g128)Backbone=Llama2-7B, Bitwidth=4.56, #codebooks=162025.02 | 77.97 | |
| GPTQAvg. Bitwidth=4.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 77.97 | |
| QuiP#Backbone=Llama2-7B, Bitwidth=4.02, #codebooks=2 * 2^162025.02 | 77.91 | |
| HPTQAvg. Bitwidth=3.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 77.8 | |
| HPTQAvg. Bitwidth=4.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 77.75 | |
| LO-BCQ (g128)Backbone=Llama2-7B, Bitwidth=4.44, #codebooks=82025.02 | 77.69 | |
| QuIP#Avg. Bitwidth=3.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 77.3 | |
| SSQR-1%Avg. Bitwidth=3.445, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 77.15 | |
| AWQ (g128)Backbone=Llama2-7B, Bitwidth=4.132025.02 | 77.09 | |
| AQLMAvg. Bitwidth=3.040, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 76.88 | |
| GPTQ (g128)Backbone=Llama2-7B, Bitwidth=42025.02 | 76.61 | |
| QTIPAvg. Bitwidth=2.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 75.9 | |
| QuIP#Avg. Bitwidth=2.000, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 75.1 | |
| AQLMAvg. Bitwidth=2.290, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 74.92 | |
| AQLMAvg. Bitwidth=2.020, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 74.76 | |
| GPTQAvg. Bitwidth=3.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 73.88 | |
| HPTQAvg. Bitwidth=2.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 73.56 | |
| GPTQAvg. Bitwidth=2.125, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 56.2 | |
| SSQR-1%Avg. Bitwidth=2.445, Backbone=Llama-2-7B, Zero-shot=true2025.07 | 56.15 |