List your company
ResearchFormerly WizWand

Question Answering on QA Zero-shot Average

73.45QA Zero-shot AverageBaseline
34.075644.297854.5264.7422Feb 5, 2026
Updated 8mo ago

Evaluation results

MethodLinks
BaselineBackbone=Gemma3-12B, Bit=162026.0273.45
QTIPBackbone=Gemma3-12B, Bit=22026.0269.69
RaBiTBackbone=Gemma3-12B, Bit=22026.0268.85
BaselineBackbone=Llama3-8B, Bit=16, Evaluation Protocol=Zero-shot2026.0268.66
DBFBackbone=Gemma3-12B, Bit=22026.0268.37
BaselineBackbone=Gemma3-4B, Bit=162026.0267.6
RaBiTBackbone=Llama3-8B, Bit=3, Evaluation Protocol=Zero-shot2026.0265.61
BaselineBackbone=Llama2-13B, Bit=16, Evaluation Protocol=Zero-shot2026.0265.46
DBFBackbone=Llama3-8B, Bit=2.3, Evaluation Protocol=Zero-shot2026.0264.84
RaBiTBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.0264.13
AQLMBackbone=Llama3-8B, Bit=2.3, Evaluation Protocol=Zero-shot2026.0264.12
RaBiTBackbone=Llama2-13B, Bit=3, Evaluation Protocol=Zero-shot2026.0264.09
QuIP#Backbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.0263.89
QTIPBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.0263.88
QTIPBackbone=Gemma3-4B, Bit=22026.0263.47
RaBiTBackbone=Llama2-7B, Bit=3, Evaluation Protocol=Zero-shot2026.0263.05
QTIPBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.0262.92
DBFBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.0262.9
MBOKBackbone=Llama2-13B, Bit=3, Evaluation Protocol=Zero-shot2026.0262.73
DBFBackbone=Llama2-13B, Bit=2.3, Evaluation Protocol=Zero-shot2026.0262.53
BaselineBackbone=Llama2-7B, Bit=16, Evaluation Protocol=Zero-shot2026.0262.26
RaBiTBackbone=Gemma3-4B, Bit=22026.0262.21
RaBiTBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.0262.1
EfficientQATBackbone=Llama2-13B, Bit=2.1, Evaluation Protocol=Zero-shot2026.0262.07
QuIP#Backbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.0261.96
AQLMBackbone=Llama2-13B, Bit=2.2, Evaluation Protocol=Zero-shot2026.0261.58
DBFBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.0261.53
RaBiTBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.0261.51
MBOKBackbone=Llama3-8B, Bit=3, Evaluation Protocol=Zero-shot2026.0261.08
BitStackBackbone=Llama2-13B, Bit=3, Evaluation Protocol=Zero-shot2026.0261.06
DBFBackbone=Gemma3-4B, Bit=22026.0260.91
EfficientQATBackbone=Llama3-8B, Bit=2.1, Evaluation Protocol=Zero-shot2026.0260.63
MBOKBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.0260.58
DBFBackbone=Llama2-7B, Bit=2.3, Evaluation Protocol=Zero-shot2026.0259.84
DB-LLMBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.0259.41
QTIPBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.0258.97
AQLMBackbone=Llama2-7B, Bit=2.3, Evaluation Protocol=Zero-shot2026.0258.57
DBFBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.0258.42
BitStackBackbone=Llama3-8B, Bit=3, Evaluation Protocol=Zero-shot2026.0258.41
QuIP#Backbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.0258.23
BaselineBackbone=Gemma3-1B, Bit=162026.0257.82
EfficientQATBackbone=Llama2-7B, Bit=2.1, Evaluation Protocol=Zero-shot2026.0257.75
BitStackBackbone=Llama2-7B, Bit=3, Evaluation Protocol=Zero-shot2026.0256.54
DB-LLMBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.0255.12
MBOKBackbone=Llama2-7B, Bit=3, Evaluation Protocol=Zero-shot2026.0254.63
MBOKBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.0254.41
MBOKBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.0253.63
RaBiTBackbone=Gemma3-1B, Bit=22026.0253.18
DBFBackbone=Gemma3-1B, Bit=22026.0251.98
DB-LLMBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.0250.92
QTIPBackbone=Gemma3-1B, Bit=22026.0250.3
GPTQBackbone=Llama2-13B, Bit=2.1, Evaluation Protocol=Zero-shot2026.0243.72
BitStackBackbone=Llama2-7B, Bit=2, Evaluation Protocol=Zero-shot2026.0240.12
BitStackBackbone=Llama2-13B, Bit=2, Evaluation Protocol=Zero-shot2026.0239.38
GPTQBackbone=Llama2-7B, Bit=2.1, Evaluation Protocol=Zero-shot2026.0239.16
BitStackBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.0236.21
GPTQBackbone=Llama3-8B, Bit=2, Evaluation Protocol=Zero-shot2026.0235.59