Question Answering on QA Zero-shot Average
73.45QA Zero-shot AverageBaseline
Evaluation Results
| Method | Links | |
|---|---|---|
| BaselineBackbone=Gemma3-12B, Bit=162026.02 | 73.45 | |
| QTIPBackbone=Gemma3-12B, Bit=22026.02 | 69.69 | |
| RaBiTBackbone=Gemma3-12B, Bit=22026.02 | 68.85 | |
| BaselineBackbone=Llama3-8B, Bit=16, Evaluation Protocol=Zero-shot2026.02 | 68.66 | |
| DBFBackbone=Gemma3-12B, Bit=22026.02 | 68.37 | |
| BaselineBackbone=Gemma3-4B, Bit=162026.02 | 67.6 | |
| RaBiTBackbone=Llama3-8B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 65.61 | |
| BaselineBackbone=Llama2-13B, Bit=16, Evaluation Protocol=Zero-shot2026.02 | 65.46 | |
| DBFBackbone=Llama3-8B, Bit=2.3, Evaluation Protocol=Zero-shot2026.02 | 64.84 | |
| RaBiTBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 64.13 | |
| AQLMBackbone=Llama3-8B, Bit=2.3, Evaluation Protocol=Zero-shot2026.02 | 64.12 | |
| RaBiTBackbone=Llama2-13B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 64.09 | |
| QuIP#Backbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 63.89 | |
| QTIPBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 63.88 | |
| QTIPBackbone=Gemma3-4B, Bit=22026.02 | 63.47 | |
| RaBiTBackbone=Llama2-7B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 63.05 | |
| QTIPBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 62.92 | |
| DBFBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 62.9 | |
| MBOKBackbone=Llama2-13B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 62.73 | |
| DBFBackbone=Llama2-13B, Bit=2.3, Evaluation Protocol=Zero-shot2026.02 | 62.53 | |
| BaselineBackbone=Llama2-7B, Bit=16, Evaluation Protocol=Zero-shot2026.02 | 62.26 | |
| RaBiTBackbone=Gemma3-4B, Bit=22026.02 | 62.21 | |
| RaBiTBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 62.1 | |
| EfficientQATBackbone=Llama2-13B, Bit=2.1, Evaluation Protocol=Zero-shot2026.02 | 62.07 | |
| QuIP#Backbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 61.96 | |
| AQLMBackbone=Llama2-13B, Bit=2.2, Evaluation Protocol=Zero-shot2026.02 | 61.58 | |
| DBFBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 61.53 | |
| RaBiTBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 61.51 | |
| MBOKBackbone=Llama3-8B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 61.08 | |
| BitStackBackbone=Llama2-13B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 61.06 | |
| DBFBackbone=Gemma3-4B, Bit=22026.02 | 60.91 | |
| EfficientQATBackbone=Llama3-8B, Bit=2.1, Evaluation Protocol=Zero-shot2026.02 | 60.63 | |
| MBOKBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 60.58 | |
| DBFBackbone=Llama2-7B, Bit=2.3, Evaluation Protocol=Zero-shot2026.02 | 59.84 | |
| DB-LLMBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 59.41 | |
| QTIPBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 58.97 | |
| AQLMBackbone=Llama2-7B, Bit=2.3, Evaluation Protocol=Zero-shot2026.02 | 58.57 | |
| DBFBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 58.42 | |
| BitStackBackbone=Llama3-8B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 58.41 | |
| QuIP#Backbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 58.23 | |
| BaselineBackbone=Gemma3-1B, Bit=162026.02 | 57.82 | |
| EfficientQATBackbone=Llama2-7B, Bit=2.1, Evaluation Protocol=Zero-shot2026.02 | 57.75 | |
| BitStackBackbone=Llama2-7B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 56.54 | |
| DB-LLMBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 55.12 | |
| MBOKBackbone=Llama2-7B, Bit=3, Evaluation Protocol=Zero-shot2026.02 | 54.63 | |
| MBOKBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 54.41 | |
| MBOKBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 53.63 | |
| RaBiTBackbone=Gemma3-1B, Bit=22026.02 | 53.18 | |
| DBFBackbone=Gemma3-1B, Bit=22026.02 | 51.98 | |
| DB-LLMBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 50.92 | |
| QTIPBackbone=Gemma3-1B, Bit=22026.02 | 50.3 | |
| GPTQBackbone=Llama2-13B, Bit=2.1, Evaluation Protocol=Zero-shot2026.02 | 43.72 | |
| BitStackBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 40.12 | |
| BitStackBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 39.38 | |
| GPTQBackbone=Llama2-7B, Bit=2.1, Evaluation Protocol=Zero-shot2026.02 | 39.16 | |
| BitStackBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 36.21 | |
| GPTQBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.02 | 35.59 |