List your company
ResearchFormerly WizWand

Zero-shot Reasoning on HellaSwag (200 items subset)

68AccuracyHQMQ s192 r6 + Med3x
20.6832.96545.2557.535May 12, 2026May 14, 2026May 16, 2026May 19, 2026May 21, 2026May 23, 2026May 26, 2026
Updated 5mo ago

Evaluation results

MethodLinks
HQMQ s192 r6 + Med3xbits=5.152026.0568
HQMQ s96 r6 + Med3xbits=4.912026.0565
HQMQ s24 r6 + Med3xbits=4.422026.0564
fp16bits=16.002026.0563
fp16 baselineCompression=1×, Base Model=TinyLlama-1.1B2026.0549.5
FIBQUANTb (bits/coord)=3.50, Compression=4.6×, k=2, N=128, Base Model=TinyLlama-1.1B2026.0548
INTb (bits/coord)=4.00, Compression=4×, Base Model=TinyLlama-1.1B2026.0547.5
FIBQUANTb (bits/coord)=4.00, Compression=4×, k=2, N=256, Base Model=TinyLlama-1.1B2026.0547.5
FIBQUANTb (bits/coord)=3.00, Compression=5.3×, k=2, N=64, Base Model=TinyLlama-1.1B2026.0547.5
TURBOQUANTb (bits/coord)=3.00, Compression=5.3×, Base Model=TinyLlama-1.1B2026.0546.5
TURBOQUANTb (bits/coord)=4.00, Compression=4×, Base Model=TinyLlama-1.1B2026.0545.5
FIBQUANTb (bits/coord)=2.50, Compression=6.4×, k=4, N=1024, Base Model=TinyLlama-1.1B2026.0544
FIBQUANTb (bits/coord)=2.00, Compression=8×, k=4, N=256, Base Model=TinyLlama-1.1B2026.0541.5
INTb (bits/coord)=3.00, Compression=5.3×, Base Model=TinyLlama-1.1B2026.0537
TURBOQUANTb (bits/coord)=2.00, Compression=8×, Base Model=TinyLlama-1.1B2026.0533.5
naive int4bits=4.002026.0530
INTb (bits/coord)=2.00, Compression=8×, Base Model=TinyLlama-1.1B2026.0522.5