List your company
ResearchFormerly WizWand

Zero-shot Commonsense Reasoning on PIQA zero-shot

76.93AccuracyFP16
53.446859.543465.6471.7366Apr 21, 2026May 1, 2026May 12, 2026May 23, 2026Jun 3, 2026Jun 14, 2026Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
FP16Bits=–, Backbone=LLAMA-2-7B2026.0476.93
TriLM-3.9B#Tokens=300B, Quantization=W1.58A162026.0674.44
BitNetV1-7B#Tokens=100B, Quantization=W1.58A162026.0674.37
TequilaLLM-3B#Tokens=10B, Quantization=W1.58A162026.0673.9
TernaryLLM-8B+KD#Tokens=1T, Quantization=W1.58A162026.0673.7
Qwen3-8B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.0673.45
BitNetV1-3.9B#Tokens=100B, Quantization=W1.58A162026.0673.2
FBI-LLMBits=W1A16, Backbone=LLAMA-2-7B2026.0472.6
Qwen3-4B + CAT-Q#Tokens=2M, Quantization=W1.58A162026.0671.55
LBLLMBits=W(1+1)A4, Backbone=LLAMA-2-7B2026.0470.97
Original EndpointModel=1B, Zero-shot evaluation=true2026.0670.95
Qwen3-4B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.0670.62
TriLM-1.5B#Tokens=300B, Quantization=W1.58A162026.0670.3
BitNetV1-1.3B#Tokens=100B, Quantization=W1.58A162026.0668.8
Qwen3-1.7B + CAT-Q#Tokens=1M, Quantization=W1.58A162026.0668.4
OnebitBits=W1A16, Backbone=LLAMA-2-7B2026.0468.01
Original PythiaModel=410M, Zero-shot evaluation=true2026.0666.43
Yat (pn+α)Parameterization=per-neuron, Scaling=learnable per-channel, Evaluation mode=zero-shot2026.0562.44
Yat (sb+α)Parameterization=shared (b, ε), Scaling=learnable per-channel, Evaluation mode=zero-shot2026.0562.37
GELUArchitecture=GELU MLP, Evaluation mode=zero-shot2026.0561.77
Yat (sb+ca)Parameterization=shared (b, ε), Scaling=constant per-channel, Evaluation mode=zero-shot2026.0561.5
Yat (pn+ca)Parameterization=per-neuron, Scaling=constant per-channel, Evaluation mode=zero-shot2026.0561.26
Dual Learned MatchingModel=1B, Zero-shot evaluation=true2026.0656.42
Learned MatchingModel=1B, Zero-shot evaluation=true2026.0655.98
Dual Learned MatchingModel=410M, Zero-shot evaluation=true2026.0655.71
Learned MatchingModel=410M, Zero-shot evaluation=true2026.0655.44
Raw InterpolationModel=410M, Zero-shot evaluation=true2026.0655.39
Raw InterpolationModel=1B, Zero-shot evaluation=true2026.0654.35