List your company
ResearchFormerly WizWand

Commonsense Reasoning on WinoGrande (WG)

77.5AccuracyQwen3-8B
49.596856.840964.08571.3291Jun 11, 2026Jun 14, 2026Jun 18, 2026Jun 21, 2026Jun 25, 2026Jun 28, 2026Jul 2, 2026
Updated 4mo ago

Evaluation results

MethodLinks
Qwen3-8BType=AR2026.0677.5
TWLABackbone=LLaMA2-70B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0677.35
Qwen2.5-7BType=AR2026.0676.1
MiMo-7BType=AR2026.0675.9
Dream-v0-7BType=Diffusion2026.0674.5
LLaDA-8BType=Diffusion2026.0674.5
DreamReasoner-8BType=Block Diffusion2026.0674.2
OLMo3-7BType=AR2026.0673.5
FP16Bits=16.02026.0772.8
HQQBits=4.02026.0772.3
TASA b4.0Bits=4.02026.0771.9
AWQBits=4.02026.0771.3
TWLABackbone=Qwen3-32B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0671.27
GPTQBits=4.02026.0770.9
TWLABackbone=Qwen3-14B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0670.64
SpQR†Bits=4.12 (W3)2026.0770.6
RTNBits=4.02026.0770.5
TWLABackbone=LLaMA2-13B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0668.75
TASA b3.0Bits=3.02026.0768.4
TWLABackbone=LLaMA3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0668.35
HQQBits=3.02026.0768.2
OWQBits=3.02026.0766.4
GPTQBits=3.02026.0765.7
TWLABackbone=LLaMA2-7B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0665.19
AuroraModel Size=1.1B, Evaluation Protocol=5-shot2026.0664.9
NorMuonModel Size=1.1B, Evaluation Protocol=5-shot2026.0664.8
TWLABackbone=Qwen3-8B, Weight bits=1.58, Activation bits=16, Setting=Zero-shot2026.0664.72
MuonModel Size=1.1B, Evaluation Protocol=5-shot2026.0663.1
U-NorMuonModel Size=1.1B, Evaluation Protocol=5-shot2026.0661.9
RTNBits=3.02026.0756.1
NorMuonModel Size=340M, Evaluation Protocol=5-shot2026.0654.9
U-NorMuonModel Size=340M, Evaluation Protocol=5-shot2026.0654.9
AuroraModel Size=340M, Evaluation Protocol=5-shot2026.0654.4
MuonModel Size=340M, Evaluation Protocol=5-shot2026.0651.7
ADAMS (DENSE all-reduce)Protocol=Zero-shot, Model=GPT-345M, Strategy=DENSE all-reduce2026.0751.54
ADAMW (DENSE all-reduce)Protocol=Zero-shot, Model=GPT-345M, Strategy=DENSE all-reduce2026.0751.38
SCAPE (d = 0.1)Protocol=Zero-shot, Model=GPT-345M, d=0.12026.0751.38
SCAPE (d = 0.01)Protocol=Zero-shot, Model=GPT-345M, d=0.012026.0750.67