List your company
ResearchFormerly WizWand

Commonsense Reasoning on WinoGrande (val)

73.88AccuracyFP16
47.443254.306661.1768.0334May 7, 2024Aug 23, 2024Dec 9, 2024Mar 28, 2025Jul 14, 2025Oct 30, 2025Feb 16, 2026
Updated 8mo ago

Evaluation results

MethodLinks
FP16Backbone=Mistral-7b, Bits Per FPN=162024.0573.88
KVQuant-4bBackbone=Mistral-7b, Bits Per FPN=4.00-4.022024.0573.88
KVQuant-4b-1%Backbone=Mistral-7b, Bits Per FPN=4.32-4.352024.0573.72
CQ-2c8bBackbone=Mistral-7b, Bits Per FPN=4.002024.0573.48
KVQuant-4b-1%Backbone=LLaMA-13b, Bits Per FPN=4.32-4.352024.0573.4
FP16Backbone=LLaMA-13b, Bits Per FPN=162024.0572.69
KVQuant-4bBackbone=LLaMA-13b, Bits Per FPN=4.00-4.022024.0572.61
CQ-2c8bBackbone=LLaMA-2-13b, Bits Per FPN=4.002024.0572.53
CQ-2c8bBackbone=LLaMA-13b, Bits Per FPN=4.002024.0572.45
KVQuant-4b-1%Backbone=LLaMA-2-13b, Bits Per FPN=4.32-4.352024.0572.3
FP16Backbone=LLaMA-2-13b, Bits Per FPN=162024.0571.98
KVQuant-4bBackbone=LLaMA-2-13b, Bits Per FPN=4.00-4.022024.0571.59
KVQuant-2b-1%Backbone=LLaMA-13b, Bits Per FPN=2.32-2.352024.0571.43
KVQuant-2b-1%Backbone=Mistral-7b, Bits Per FPN=2.32-2.352024.0570.8
KVQuant-4b-1%Backbone=LLaMA-7b, Bits Per FPN=4.32-4.352024.0570.72
CQ-4c8bBackbone=LLaMA-13b, Bits Per FPN=2.002024.0570.72
CQ-2c8bBackbone=LLaMA-7b, Bits Per FPN=4.002024.0570.4
KVQuant-2b-1%Backbone=LLaMA-2-13b, Bits Per FPN=2.32-2.352024.0570.17
FP16Backbone=LLaMA-7b, Bits Per FPN=162024.0569.93
KVQuant-4bBackbone=LLaMA-7b, Bits Per FPN=4.00-4.022024.0569.53
CQ-4c8bBackbone=Mistral-7b, Bits Per FPN=2.002024.0569.38
CQ-4c8bBackbone=LLaMA-2-13b, Bits Per FPN=2.002024.0569.06
FP16Backbone=LLaMA-2-7b, Bits Per FPN=162024.0568.9
Qwen-1.5 14B (Teacher)Model Scale=14B2024.0768.74
KVQuant-4b-1%Backbone=LLaMA-2-7b, Bits Per FPN=4.32-4.352024.0568.67
CQ-2c8bBackbone=LLaMA-2-7b, Bits Per FPN=4.002024.0568.27
KVQuant-2b-1%Backbone=LLaMA-7b, Bits Per FPN=2.32-2.352024.0568.03
KVQuant-4bBackbone=LLaMA-2-7b, Bits Per FPN=4.00-4.022024.0567.96
KVQuant-2b-1%Backbone=LLaMA-2-7b, Bits Per FPN=2.32-2.352024.0567.64
CQ-4c8bBackbone=LLaMA-7b, Bits Per FPN=2.002024.0567.48
CQ-4c8bBackbone=LLaMA-2-7b, Bits Per FPN=2.002024.0566.45
CQ-8c10bBackbone=LLaMA-13b, Bits Per FPN=1.252024.0565.27
D2-LoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.0264
CQ-8c10bBackbone=Mistral-7b, Bits Per FPN=1.252024.0563.93
LLaMA2-13BModel Size=13B, Shots=0-shot2024.0763.77
LoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.0263.6
KVQuant-2bBackbone=Mistral-7b, Bits Per FPN=2.00-2.022024.0563.46
D2-LoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.0263
CQ-8c10bBackbone=LLaMA-2-13b, Bits Per FPN=1.252024.0562.98
DoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.0262.6
DoRABackbone=Qwen2.5-7B-Instruct, Training Epochs=1, Scoring Method=CLL2026.0261.6
CQ-8c8bBackbone=LLaMA-13b, Bits Per FPN=1.002024.0561.56
KVQuant-1b-1%Backbone=LLaMA-13b, Bits Per FPN=1.32-1.352024.0561.01
CQ-8c10bBackbone=LLaMA-7b, Bits Per FPN=1.252024.0560.46
KVQuant-2bBackbone=LLaMA-13b, Bits Per FPN=2.00-2.022024.0559.35
LoRABackbone=Llama-3.2-3B-Instruct, Training Epochs=2, Scoring Method=CLL2026.0259.2
CQ-8c10bBackbone=LLaMA-2-7b, Bits Per FPN=1.252024.0559.19
Qwen-1.5 1.8B + DDKDistillation=DDK2024.0759.1
CQ-8c8bBackbone=Mistral-7b, Bits Per FPN=1.002024.0558.25
KVQuant-1b-1%Backbone=Mistral-7b, Bits Per FPN=1.32-1.352024.0558.17
Qwen-1.5 1.8B + KDDistillation=KD2024.0758.01
Qwen-1.5 1.8B (Student)Model Scale=1.8B2024.0757.85
KVQuant-1b-1%Backbone=LLaMA-2-7b, Bits Per FPN=1.32-1.352024.0557.77
TinyLLaMA-1.1B + DDKModel Size=1.1B, Shots=0-shot2024.0757.62
Qwen-1.5 1.8B + TEDDistillation=TED2024.0757.38
KVQuant-1b-1%Backbone=LLaMA-2-13b, Bits Per FPN=1.32-1.352024.0557.3
CQ-8c8bBackbone=LLaMA-2-13b, Bits Per FPN=1.002024.0557.14
Qwen-1.5 1.8B + MiniLLMDistillation=MiniLLM2024.0757.14
Qwen-1.5 1.8B + CPTDistillation=CPT2024.0756.98
Qwen-1.5 1.8B + CPT & DoReMiDistillation=CPT & DoReMi2024.0756.75
KVQuant-1b-1%Backbone=LLaMA-7b, Bits Per FPN=1.32-1.352024.0556.67
CQ-8c8bBackbone=LLaMA-7b, Bits Per FPN=1.002024.0556.51
TinyLLaMA-1.1B + CPTModel Size=1.1B, Shots=0-shot2024.0756.2
TinyLLaMA-1.1B + TEDModel Size=1.1B, Shots=0-shot2024.0755.72
TinyLLaMA-1.1BModel Size=1.1B, Shots=0-shot2024.0755.49
TinyLLaMA-1.1B + CPT & DoReMiModel Size=1.1B, Shots=0-shot2024.0755.25
CQ-8c8bBackbone=LLaMA-2-7b, Bits Per FPN=1.002024.0555.01
TinyLLaMA-1.1B + MiniLLMModel Size=1.1B, Shots=0-shot2024.0754.46
Mistral (Full-Attention)Model Scale=1.4B2024.0754.4
TinyLLaMA-1.1B + KDModel Size=1.1B, Shots=0-shot2024.0753.91
KVQuant-2bBackbone=LLaMA-7b, Bits Per FPN=2.00-2.022024.0553.59
BMoJo (Fading)Model Scale=1.4B2024.0753.3
Hybrid (Sliding Attention + SSM)Model Scale=1.4B2024.0752.6
BMoJo (Fading + Eidetic)Model Scale=1.4B2024.0752.1
Hybrid (Sliding Attention + SSM)Model Scale=370M2024.0751.9
Mamba (SSM)Model Scale=1.4B2024.0751.9
BMoJo (Fading)Model Scale=370M2024.0751.8
KVQuant-2bBackbone=LLaMA-2-7b, Bits Per FPN=2.00-2.022024.0551.7
Mamba (SSM)Model Scale=370M2024.0751.7
KVQuant-2bBackbone=LLaMA-2-13b, Bits Per FPN=2.00-2.022024.0551.3
KVQuant-1bBackbone=LLaMA-2-7b, Bits Per FPN=1.00-1.022024.0550.91
BMoJo (Fading + Eidetic)Model Scale=370M2024.0750.7
KVQuant-1bBackbone=LLaMA-7b, Bits Per FPN=1.00-1.022024.0550.51
Mistral (Full-Attention)Model Scale=370M2024.0750.4
KVQuant-1bBackbone=Mistral-7b, Bits Per FPN=1.00-1.022024.0549.8
KVQuant-1bBackbone=LLaMA-2-13b, Bits Per FPN=1.00-1.022024.0549.41
KVQuant-1bBackbone=LLaMA-13b, Bits Per FPN=1.00-1.022024.0548.46