List your company

Zero-shot Evaluation on Suite 0-shot

70.97AccuracyFP
30.30640.86351.4261.977May 21, 2026

Evaluation results

MethodLinks
FP#Bits=16.0, Backbone=Mixtral-8×7B2026.0570.97
GEMQ#Bits=2.5, Backbone=Mixtral-8×7B2026.0565.13
SpQR#Bits=2.5, Backbone=Mixtral-8×7B2026.0564.92
GEMQ#Bits=1.5, Backbone=Mixtral-8×7B2026.0552
SpQR#Bits=1.5, Backbone=Mixtral-8×7B2026.0531.87