List your company
ResearchFormerly WizWand

Accuracy on PiQA (Question Answering)

81.77AccuracyLO-BCQ (g128)
55.125262.042668.9675.8774Feb 7, 2025Mar 6, 2025Apr 3, 2025May 1, 2025May 29, 2025Jun 26, 2025Jul 24, 2025
Updated 8mo ago

Evaluation results

MethodLinks
LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.31, #codebooks=42025.0281.77
AQLMBackbone=Llama2-70B, Bitwidth=4.14, #codebooks=2 * 2^162025.0281.5
LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.44, #codebooks=82025.0281.5
QuiP#Backbone=Llama2-70B, Bitwidth=4.02, #codebooks=2 * 2^162025.0281.45
LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.56, #codebooks=162025.0281.45
GPTQ (g128)Backbone=Llama2-70B, Bitwidth=42025.0281.23
LO-BCQ (g128)Backbone=Llama2-70B, Bitwidth=4.19, #codebooks=22025.0281.18
QuIP#Avg. Bitwidth=4.000, Backbone=Llama-2-7B, Zero-shot=true2025.0778.4
QTIPAvg. Bitwidth=4.000, Backbone=Llama-2-7B, Zero-shot=true2025.0778.4
SSQR-1%Avg. Bitwidth=4.445, Backbone=Llama-2-7B, Zero-shot=true2025.0778.35
LO-BCQ (g128)Backbone=Llama2-7B, Bitwidth=4.19, #codebooks=22025.0278.29
AQLMAvg. Bitwidth=5.020, Backbone=Llama-2-7B, Zero-shot=true2025.0778.29
AQLMBackbone=Llama2-7B, Bitwidth=4.14, #codebooks=2 * 2^162025.0278.24
AQLMAvg. Bitwidth=4.040, Backbone=Llama-2-7B, Zero-shot=true2025.0778.24
LO-BCQ (g128)Backbone=Llama2-7B, Bitwidth=4.31, #codebooks=42025.0278.18
FP16 BaselineAvg. Bitwidth=16.000, Backbone=Llama-2-7B, Zero-shot=true2025.0778.13
QTIPAvg. Bitwidth=3.000, Backbone=Llama-2-7B, Zero-shot=true2025.0778.1
LO-BCQ (g128)Backbone=Llama2-7B, Bitwidth=4.56, #codebooks=162025.0277.97
GPTQAvg. Bitwidth=4.125, Backbone=Llama-2-7B, Zero-shot=true2025.0777.97
QuiP#Backbone=Llama2-7B, Bitwidth=4.02, #codebooks=2 * 2^162025.0277.91
HPTQAvg. Bitwidth=3.125, Backbone=Llama-2-7B, Zero-shot=true2025.0777.8
HPTQAvg. Bitwidth=4.125, Backbone=Llama-2-7B, Zero-shot=true2025.0777.75
LO-BCQ (g128)Backbone=Llama2-7B, Bitwidth=4.44, #codebooks=82025.0277.69
QuIP#Avg. Bitwidth=3.000, Backbone=Llama-2-7B, Zero-shot=true2025.0777.3
SSQR-1%Avg. Bitwidth=3.445, Backbone=Llama-2-7B, Zero-shot=true2025.0777.15
AWQ (g128)Backbone=Llama2-7B, Bitwidth=4.132025.0277.09
AQLMAvg. Bitwidth=3.040, Backbone=Llama-2-7B, Zero-shot=true2025.0776.88
GPTQ (g128)Backbone=Llama2-7B, Bitwidth=42025.0276.61
QTIPAvg. Bitwidth=2.000, Backbone=Llama-2-7B, Zero-shot=true2025.0775.9
QuIP#Avg. Bitwidth=2.000, Backbone=Llama-2-7B, Zero-shot=true2025.0775.1
AQLMAvg. Bitwidth=2.290, Backbone=Llama-2-7B, Zero-shot=true2025.0774.92
AQLMAvg. Bitwidth=2.020, Backbone=Llama-2-7B, Zero-shot=true2025.0774.76
GPTQAvg. Bitwidth=3.125, Backbone=Llama-2-7B, Zero-shot=true2025.0773.88
HPTQAvg. Bitwidth=2.125, Backbone=Llama-2-7B, Zero-shot=true2025.0773.56
GPTQAvg. Bitwidth=2.125, Backbone=Llama-2-7B, Zero-shot=true2025.0756.2
SSQR-1%Avg. Bitwidth=2.445, Backbone=Llama-2-7B, Zero-shot=true2025.0756.15