Recognizing Textual Entailment on RTE (Accuracy)
83.13AccuracyELSA
Evaluation results
| Method | Links | |
|---|---|---|
| ELSAalpha_hat=0.22026.01 | 83.13 | |
| RoFedalpha_hat=0.22026.01 | 82.33 | |
| FedCAdaalpha_hat=0.22026.01 | 81.31 | |
| FedAMSalpha_hat=0.22026.01 | 81.22 | |
| ELSAalpha_hat=0.12026.01 | 80.93 | |
| RaSAalpha_hat=0.22026.01 | 80.53 | |
| FedAMSalpha_hat=0.12026.01 | 80.13 | |
| FedAvgalpha_hat=0.22026.01 | 79.91 | |
| FedProxalpha_hat=0.22026.01 | 79.54 | |
| FedCAdaalpha_hat=0.12026.01 | 79.23 | |
| FedAvgalpha_hat=0.12026.01 | 79.21 | |
| RoFedalpha_hat=0.12026.01 | 79.18 | |
| FedAvg (Random)alpha_hat=0.22026.01 | 79.03 | |
| FedProxalpha_hat=0.12026.01 | 78.92 | |
| RaSAalpha_hat=0.12026.01 | 78.92 | |
| FedAvg (Random)alpha_hat=0.12026.01 | 78.48 | |
| Swimba-14BParameters=14B, Experts=4, FLOPs / token=1.51282 × 10^102026.03 | 73.2 | |
| Nemotron-H-8BParameters=8B, FLOPs / token=1.51263 × 10^102026.03 | 71.8 | |
| GRAPEModel=Mixtral-8x22B, Scope=Global, Number of experts pruned (e)=2e2026.04 | 71.4 | |
| OriginalModel=Mixtral-8x22B, Scope=None2026.04 | 71.2 | |
| DEKModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 71.1 | |
| EnumerateModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 70.1 | |
| Count-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 69.3 | |
| GRAPEModel=Mixtral-8x22B, Scope=Global, Number of experts pruned (e)=4e2026.04 | 68.5 | |
| DEKModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 68.1 | |
| No pruningSparsity=0, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 67.9 | |
| EnumerateModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 67.9 | |
| DenseBase Model=DeepSeek-7B, Sparsity=Dense2025.06 | 67.87 | |
| Count-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 67.5 | |
| CRePEModel=LLaMA-2-7B, Sparsity Pattern=unstructured 50%, Evaluation Protocol=Zero-shot2026.06 | 66.43 | |
| DEKModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 66.4 | |
| MaskProBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 66.32 | |
| OriginalModel=DeepSeek-MoE, Scope=None2026.04 | 66 | |
| GRAPEModel=DeepSeek-MoE, Scope=Global, Number of experts pruned (e)=4e2026.04 | 65.5 | |
| BF16Backbone=Llama-2-13B2026.05 | 65.34 | |
| DenseModel=LLaMA-2-7B, Sparsity Pattern=Dense (None), Evaluation Protocol=Zero-shot2026.06 | 65.34 | |
| GRAPEModel=DeepSeek-MoE, Scope=Global, Number of experts pruned (e)=2e2026.04 | 65 | |
| Count-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 64.9 | |
| DEKModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 64.6 | |
| PuDDingSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 64.3 | |
| SparsegptBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 63.43 | |
| Pruner-ZBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 63.15 | |
| BF16Backbone=LLaMA-2-7B2026.05 | 62.82 | |
| DenseBase Model=LLaMA-2-7B, Sparsity=Dense2025.06 | 62.82 | |
| Magnitude-DimSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 62.1 | |
| DIETSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 62.1 | |
| MaskProBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 61.37 | |
| SparseGPTModel=LLaMA-2-7B, Sparsity Pattern=unstructured 50%, Evaluation Protocol=Zero-shot2026.06 | 61.01 | |
| Router-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=2e2026.04 | 60.4 | |
| Router-guidedModel=DeepSeek-MoE, Scope=Local, Number of experts pruned (e)=4e2026.04 | 60.2 | |
| Pruner-ZBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 59.12 | |
| RIAModel=LLaMA-2-7B, Sparsity Pattern=unstructured 50%, Evaluation Protocol=Zero-shot2026.06 | 58.84 | |
| SparsegptBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 58.48 | |
| CRePEModel=LLaMA-2-7B, Sparsity Pattern=2:4, Evaluation Protocol=Zero-shot2026.06 | 57.76 | |
| PuDDingSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 57.4 | |
| Count-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 57.4 | |
| LGPBackbone=Llama-2-13B2026.05 | 56.68 | |
| SparseGPTModel=LLaMA-2-7B, Sparsity Pattern=2:4, Evaluation Protocol=Zero-shot2026.06 | 56.68 | |
| RIAModel=LLaMA-2-7B, Sparsity Pattern=2:4, Evaluation Protocol=Zero-shot2026.06 | 56.32 | |
| MagnitudeModel=LLaMA-2-7B, Sparsity Pattern=unstructured 50%, Evaluation Protocol=Zero-shot2026.06 | 55.96 | |
| WandaModel=LLaMA-2-7B, Sparsity Pattern=2:4, Evaluation Protocol=Zero-shot2026.06 | 55.96 | |
| LGPBackbone=LLaMA-2-7B2026.05 | 55.23 | |
| OmniQuantBackbone=Llama-2-13B2026.05 | 55.23 | |
| GPTQBackbone=LLaMA-2-7B2026.05 | 54.51 | |
| OmniQuantBackbone=LLaMA-2-7B2026.05 | 54.51 | |
| WandaModel=LLaMA-2-7B, Sparsity Pattern=unstructured 50%, Evaluation Protocol=Zero-shot2026.06 | 54.51 | |
| DIETSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 54.2 | |
| Router-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=2e2026.04 | 54.2 | |
| Magnitude-DimSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 53.8 | |
| BF16Backbone=Qwen-3-0.6B2026.05 | 53.79 | |
| MagnitudeModel=LLaMA-2-7B, Sparsity Pattern=2:4, Evaluation Protocol=Zero-shot2026.06 | 53.79 | |
| OmniQuantBackbone=Qwen-3-0.6B2026.05 | 52.71 | |
| LGPBackbone=Qwen-3-0.6B2026.05 | 52.71 | |
| SliceGPTSparsity=20%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 52.7 | |
| SliceGPTSparsity=10%, Backbone=Gemma-2 9B, Evaluation Protocol=Zero-shot2026.03 | 52.7 | |
| GPTQBackbone=Qwen-3-0.6B2026.05 | 51.62 | |
| GPTQBackbone=Llama-2-13B2026.05 | 50.18 | |
| Router-guidedModel=Mixtral-8x22B, Scope=Local, Number of experts pruned (e)=4e2026.04 | 49.5 |