Zero-shot Classification on Seven Tasks Zero-Shot
76.1Average AccuracyGEMQ
Evaluation results
| Method | Links | |
|---|---|---|
| GEMQBackbone=Mixtral-8x7B, #Bits (W-A)=3-16, Protocol=EAQuant task set2026.05 | 76.1 | |
| GEMQBackbone=Mixtral-8x7B, #Bits (W-A)=2.5-16, Protocol=EAQuant task set2026.05 | 72.52 | |
| EAQuantBackbone=Mixtral-8x7B, #Bits (W-A)=3-4, Protocol=EAQuant task set2026.05 | 71.23 | |
| GEMQBackbone=Mixtral-8x7B, #Bits (W-A)=2.5-16, Protocol=SpQR task set2026.05 | 65.13 | |
| SpQRBackbone=Mixtral-8x7B, #Bits (W-A)=2.5-16, Protocol=SpQR task set2026.05 | 64.92 | |
| GEMQBackbone=Mixtral-8x7B, #Bits (W-A)=3-16, Protocol=MoEQuant task set2026.05 | 59.49 | |
| MoEQuantBackbone=Mixtral-8x7B, #Bits (W-A)=3-16, Protocol=MoEQuant task set2026.05 | 57.24 | |
| GEMQBackbone=Mixtral-8x7B, #Bits (W-A)=1.5-16, Protocol=SpQR task set2026.05 | 52 | |
| SpQRBackbone=Mixtral-8x7B, #Bits (W-A)=1.5-16, Protocol=SpQR task set2026.05 | 31.87 |