Zero-shot Question Answering on (ARC-C, ARC-E, PIQA, Wino)
43.43ARC-C AccuracyDense
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DenseAvg. bits=16, Backbone=Llama2-7B2025.05 | 43.43 | 76.34 | 78.07 | 69.06 | 66.73 | |
| QTIPAvg. bits=2, Backbone=Llama2-7B2025.05 | 39.76 | 73.31 | 76.27 | 67.2 | 64.14 | |
| DBF + PVAvg. bits=2.3, Backbone=Llama2-7B2025.05 | 39.33 | 73.65 | 76.55 | 67.08 | 64.15 | |
| AQLM + PVAvg. bits=2.3, Backbone=Llama2-7B2025.05 | 38.91 | 72.9 | 77.37 | 67.72 | 64.23 | |
| DBFAvg. bits=2.3, Backbone=Llama2-7B2025.05 | 38.39 | 73.35 | 76.49 | 66.85 | 63.77 | |
| QUIP#Avg. bits=2, Backbone=Llama2-7B2025.05 | 37.79 | 71.88 | 75.46 | 65.66 | 62.7 | |
| DBFAvg. bits=2, Backbone=Llama2-7B2025.05 | 37.03 | 72.05 | 76.22 | 65.82 | 62.78 | |
| DBF + PVAvg. bits=2, Backbone=Llama2-7B2025.05 | 36.94 | 71.88 | 76.16 | 66.06 | 62.76 | |
| CalderaAvg. bits=2.4, Backbone=Llama2-7B2025.05 | 35.9 | 64.6 | 76.5 | 65.7 | 60.68 | |
| DBF + PVAvg. bits=1.5, Backbone=Llama2-7B2025.05 | 35.58 | 67.12 | 73.72 | 63.69 | 60.03 | |
| CalderaAvg. bits=2.1, Backbone=Llama2-7B2025.05 | 35.4 | 63.3 | 75.4 | 64.6 | 59.68 | |
| DBFAvg. bits=1.5, Backbone=Llama2-7B2025.05 | 35.06 | 66.32 | 73.28 | 64.4 | 59.77 | |
| OneBitAvg. bits=1, Backbone=Llama2-7B2025.05 | 29.61 | 41.58 | 68.12 | 58.41 | 49.43 | |
| AQLM + PVAvg. bits=1.5, Backbone=Llama2-7B2025.05 | 29.44 | 64.14 | 73.12 | 63.38 | 57.52 | |
| DBF + PVAvg. bits=1, Backbone=Llama2-7B2025.05 | 27.47 | 57.87 | 68.66 | 58.4 | 53.1 | |
| DBFAvg. bits=1, Backbone=Llama2-7B2025.05 | 26.45 | 55.3 | 67.41 | 58.87 | 52.01 | |
| BiLLMAvg. bits=1.1, Backbone=Llama2-7B2025.05 | 24.4 | 36.2 | 60.6 | 52.4 | 43.4 |