Recognizing Textual Entailment on RTE (test)
76.53AccuracyQWEN3-14B
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| QWEN3-14BModel Architecture=QWEN3-14B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 76.53 | — | |
| QWEN3-4BModel Architecture=QWEN3-4B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 76.53 | — | |
| MISTRAL-7BModel Architecture=MISTRAL-7B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 75.81 | — | |
| DEEPSEEK-R1-QWEN-8BModel Architecture=DEEPSEEK-R1-QWEN-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 74.37 | — | |
| DEEPSEEK-R1-LLAMA-8BModel Architecture=DEEPSEEK-R1-LLAMA-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 72.2 | — | |
| LLAMA-3.1-8BModel Architecture=LLAMA-3.1-8B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 71.12 | — | |
| DenseBackbone=RoBERTa2024.04 | 66.79 | — | |
| NEPENTHEBackbone=RoBERTa2024.04 | 66.06 | — | |
| Smallest weightsBackbone=RoBERTa2024.04 | 65.81 | — | |
| DARTModel Architecture=DEEPSEEK-R1-QWEN-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 63.54 | — | |
| DARTModel Architecture=MISTRAL-7B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 63.18 | — | |
| IMPBackbone=RoBERTa2024.04 | 62.82 | — | |
| DARTModel Architecture=DEEPSEEK-R1-LLAMA-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 62.45 | — | |
| Smallest gradientsBackbone=RoBERTa2024.04 | 62.2 | — | |
| DenseBackbone=BERT2024.04 | 61.01 | — | |
| NEPENTHEBackbone=BERT2024.04 | 58.12 | — | |
| IMPBackbone=BERT2024.04 | 57.76 | — | |
| EGPBackbone=BERT2024.04 | 57.73 | — | |
| DARTModel Architecture=LLAMA-3.1-8B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 55.23 | — | |
| DARTModel Architecture=LLAMA-3.2-3B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 55.23 | — | |
| Smallest gradientsBackbone=BERT2024.04 | 55.23 | — | |
| DARTModel Architecture=QWEN3-14B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 54.15 | — | |
| LLAMA-3.2-3BModel Architecture=LLAMA-3.2-3B, Configuration Type=Dense, Evaluation Protocol=Zero-shot, Sparsity Level=0%2026.01 | 54.15 | — | |
| DARTModel Architecture=QWEN3-4B, Configuration Type=Sparse, Evaluation Protocol=Zero-shot, Sparsity Level=70%2026.01 | 52.71 | — | |
| EGPBackbone=RoBERTa2024.04 | 52.71 | — | |
| Smallest weightsBackbone=BERT2024.04 | 46.93 | — |