List your company
ResearchFormerly WizWand

Natural Language Inference on CMNLI

79.07ScoreQwen2.5
26.217239.938653.6667.3814Mar 6, 2024Jun 23, 2024Oct 11, 2024Jan 29, 2025May 19, 2025Sep 6, 2025Dec 25, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Qwen2.52025.1279.07
Qwen32025.1271.43
Gamayun2025.1268.9
Llama3.22025.1262.55
Gemma32025.1257.14
Mamba-2.8BBackbone=Mamba-2.8B, Pruning ratio=0%2024.0335.97
GRASPCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1235.1
LaCoTrain-Free=false, Base Model=Llama 2 7B, Compression Ratio=25%2025.0534.4
UIDLTrain-Free=false, Base Model=Llama 2 7B, Compression Ratio=25%2025.0534.4
LaCoCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1234.4
ShortGPTCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1234.4
LLMPrunerTrain-Free=false, Base Model=Llama 2 7B, Compression Ratio=25%2025.0533.4
LLMPrunerCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1233.4
ShortGPTBackbone=RWKV-7B, Pruning ratio=28.1%2024.0333.11
Llama 2 7B (baseline)Base Model=Llama 2 7B, Compression Ratio=0%2025.0533
LLM-StreamlineTrain-Free=false, Base Model=Llama 2 7B, Compression Ratio=25%2025.0533
ReplaceMe (Cosine)Train-Free=true, Base Model=Llama 2 7B, Compression Ratio=25%2025.0533
ReplaceMe (LS)Train-Free=true, Base Model=Llama 2 7B, Compression Ratio=25%2025.0533
DenseCompression Ratio=0%, Post-training compensation=No, Evaluation Protocol=Zero-shot2024.1233
LLM-Streamline-FFNCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1233
LLM-Streamline-LayerCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1233
ShortGPTBackbone=Mamba-2.8B, Pruning ratio=10.9%2024.0332.95
ShortGPTBackbone=RWKV-7B, Pruning ratio=9.4%2024.0332.6
ShortGPTBackbone=RWKV-7B, Pruning ratio=25%2024.0332.41
ShortGPTBackbone=RWKV-7B, Pruning ratio=18.8%2024.0332.11
RWKV-7BBackbone=RWKV-7B, Pruning ratio=0%2024.0332.07
SliceGPTTrain-Free=false, Base Model=Llama 2 7B, Compression Ratio=25%2025.0531.6
SliceGPTCompression Ratio=25%, Post-training compensation=Yes, Evaluation Protocol=Zero-shot2024.1231.6
ShortGPTBackbone=Mamba-2.8B, Pruning ratio=20.3%2024.0331.29
ShortGPTBackbone=Mamba-2.8B, Pruning ratio=25%2024.0329.96
ShortGPTBackbone=Mamba-2.8B, Pruning ratio=31.3%2024.0328.25