List your company
ResearchFormerly WizWand

Reading Comprehension Accuracy on RACE

89.95AccuracyQwen-1.5 14B
30.4145.867561.32576.7825Jul 23, 2024Oct 26, 2024Jan 30, 2025May 5, 2025Aug 9, 2025Nov 12, 2025Feb 16, 2026
Updated 8mo ago

Evaluation results

MethodLinks
Qwen-1.5 14BRole=Teacher2024.0789.95
Qwen-1.5 14BModel Type=Teacher2024.0789.95
Qwen-1.5 7BRole=Teacher2024.0785.78
Qwen-1.5 7BRole=Teacher2024.0785.78
DDKTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.0782.53
DDKTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.0782.53
PoEBackbone=OpenRLHF-Llama3-8B-SFT2025.1281.39
ALBMBackbone=OpenRLHF-Llama3-8B-SFT2025.1280.69
DIRBackbone=OpenRLHF-Llama3-8B-SFT2025.1280.32
LPBackbone=OpenRLHF-Llama3-8B-SFT2025.1280.3
Qwen-1.5 4BRole=Student2024.0780.17
Qwen-1.5 4BModel Type=Student2024.0780.17
KDTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.0780.13
KDTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.0780.13
CPTTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.0779.56
CPTTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.0779.56
MiniLLMTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.0779.22
MiniLLMTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 4B2024.0779.22
BaseBackbone=OpenRLHF-Llama3-8B-SFT2025.1279.21
LPBackbone=Llama3.1-8B-Instruct2025.1278.9
SKBackbone=OpenRLHF-Llama3-8B-SFT2025.1278.82
InfoRMBackbone=OpenRLHF-Llama3-8B-SFT2025.1278.72
DDKTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 1.8B, Training Tokens=15B2024.0771.5
DDKTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 1.8B, Training Tokens=30B2024.0771.4
DDKTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 1.8B, Training Tokens=20B2024.0771.27
DDKBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.0770.98
DDKTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 1.8B, Training Tokens=10B2024.0770.84
KDBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.0769.86
Qwen-1.5 1.8BRole=Student2024.0769.57
Qwen-1.5 1.8BRole=Student2024.0769.57
DDKTeacher Model=Qwen-1.5 14B, Student Model=Qwen-1.5 1.8B, Training Tokens=5B2024.0769.4
MiniLLMBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.0769.1
CPTBase Model=Qwen-1.5 1.8B, Teacher=Qwen-1.5 7B2024.0769
BaseBackbone=Llama3.1-8B-Instruct2025.1266.5
InfoRMBackbone=Llama3.1-8B-Instruct2025.1265.2
DIRBackbone=Llama3.1-8B-Instruct2025.1262.02
PoEBackbone=Llama3.1-8B-Instruct2025.1260.03
MixtralParameters=8x22B2024.0759.2
Llama 3Parameters=70B2024.0759
ALBMBackbone=Llama3.1-8B-Instruct2025.1259
Llama 3Parameters=405B2024.0758.1
Llama 3Parameters=8B2024.0754.3
SKBackbone=Llama3.1-8B-Instruct2025.1253.89
MistralParameters=7B2024.0753
SD-MoEModel Scale=Large (7B-1.5B), Base Model=Qwen, Method Configuration=SD-MoE2026.0251.48
QwenModel Scale=Large (7B-1.5B), Base Model=Qwen, Method Configuration=Baseline2026.0250.65
SD-MoEModel Scale=Small (2B-0.8B), Base Model=Qwen, Method Configuration=SD-MoE2026.0250.61
DeepSeekModel Scale=Small (2B-0.8B), Base Model=DeepSeek, Method Configuration=Baseline2026.0250.51
SD-MoEModel Scale=Small (2B-0.8B), Base Model=DeepSeek, Method Configuration=SD-MoE2026.0250.06
GemmaParameters=7B2024.0748.8
QwenModel Scale=Small (2B-0.8B), Base Model=Qwen, Method Configuration=Baseline2026.0248.54
Nemotron-4 15B + MaskLLMBackbone=Nemotron-4 15B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.0945.45
LLaMA-2 7B (Dense)Backbone=LLaMA-2 7B, Pruning Method=Dense, Sparsity Pattern=None2024.0944.11
LLaMA-2 13B + MaskLLMBackbone=LLaMA-2 13B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.0941.24
LLaMA-2 7B + MaskLLMBackbone=LLaMA-2 7B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.0940.77
Llama3 8BCompression Ratio=Baseline2025.0940.3
Llama3 8B2026.0240.3
COMPOTCR=0.22026.0240.1
Llama 3.2 3BAttn CR=N/A2025.0840
Matrix PCABase Model=Llama 3.1 8B, Attn CR=20%2025.0840
SVD-LLMBase Model=Llama 3.1 8B, Attn CR=20%2025.0839.6
Matrix PCABase Model=Llama 3.2 3B, Attn CR=20%2025.0839.3
Llama 3.1 8BAttn CR=N/A2025.0839.3
Llama2 7BCR=None2025.0939.2
CoSpaDi (per-layer)CR=0.22025.0939
SVD-LLMBase Model=Llama 3.2 3B, Attn CR=20%2025.0838.6
COMPOTCR=0.32026.0238.4
CoSpaDi (grouped)CR=0.22025.0938.3
CoSpaDiCompression Ratio=0.22025.0938.2
DLMTokens=100B, Model Scale=1.5B, Initialization=Autoregressive LLM (Qwen2.5)2026.0137.8
Llama 3.2 1BAttn CR=N/A2025.0837.8
DLM + extra tokenTokens=100B, Model Scale=1.5B, Initialization=Autoregressive LLM (Qwen2.5)2026.0137.61
DLM + GATokens=100B, Model Scale=1.5B, Initialization=Autoregressive LLM (Qwen2.5)2026.0137.42
SVD-LLMBase Model=Llama 3.2 1B, Attn CR=20%2025.0837.3
CoSpaDi (per-layer)CR=0.32025.0936.8
SVD-LLMCR=0.22025.0936.6
COMPOTCR=0.42026.0236.6
SVD-LLMBase Model=Llama 3.2 1B, Attn CR=30%2025.0836.5
CoSpaDiCompression Ratio=0.32025.0936.2
CoSpaDi (grouped)CR=0.32025.0935.9
Matrix PCABase Model=Llama 3.2 1B, Attn CR=20%2025.0835.6
ReplaceMeCompression Ratio=0.222025.0935.4
ReplaceMeCR=0.222026.0235.4
LLM-PrunerCompression Ratio=0.22025.0935.1
LLM-PrunerCR=0.22026.0235.1
Basis SharingCR=0.22025.0934.9
SVD-LLMCR=0.32025.0934.7
Matrix PCABase Model=Llama 3.2 1B, Attn CR=30%2025.0834.2
DLM + GATokens=30B, Model Scale=0.5B, Initialization=Autoregressive LLM (Qwen2.5)2026.0134.07
ReplaceMeCompression Ratio=0.312025.0934
ReplaceMeCR=0.312026.0234
GPT3 2B + MaskLLMBackbone=GPT3 2B, Pruning Method=MaskLLM, Sparsity Pattern=2:42024.0933.88
DLM + extra tokenTokens=30B, Model Scale=0.5B, Initialization=Autoregressive LLM (Qwen2.5)2026.0133.79
Qwen3-0.6BCompression Ratio=None2026.0233.7
DLM + GATokens=100B, Initialization=from scratch, Model Scale=0.5B parameters2026.0133.68
CoSpaDi (per-layer)CR=0.42025.0933.4
CoSpaDi (grouped)CR=0.42025.0933.2
COMPOT†Compression Ratio=0.22026.0233
DLM + extra tokenTokens=100B, Initialization=from scratch, Model Scale=0.5B parameters2026.0132.73
COMPOTCompression Ratio=0.32026.0232.7