Natural Language Inference on NLI Suite (MNLI, QNLI, SNLI, RTE, SICK, SCITAIL) (test)
96.18MNLI AccuracyDARE-TIES
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| DARE-TIESSpace=Full, Backbone=LLaMA3-8B2026.06 | 96.18 | 74.11 | 94.76 | 99.19 | 83.44 | 96.64 | 90.72 | 90.36 | |
| TIESSpace=Full, Backbone=LLaMA3-8B2026.06 | 96.16 | 74.04 | 94.78 | 99.19 | 83.42 | 96.69 | 90.71 | 90.73 | |
| DARE-TIESSpace=KnOTS, Backbone=LLaMA3-8B2026.06 | 95.5 | 81.71 | 92.1 | 100.81 | 91.3 | 97.22 | 93.11 | 93.5 | |
| Task ArithmeticFramework=KnOTS, Backbone=Llama-3 8B2026.04 | 95.28 | 68.71 | 93.57 | 100 | 87.96 | 96.73 | 90.38 | — | |
| TIESSpace=KnOTS, Backbone=LLaMA3-8B2026.06 | 95.05 | 80.43 | 91.66 | 101.61 | 90.34 | 97.32 | 92.74 | 92.9 | |
| CtMSpace=CtM, Merging Rule=DARE-TIES, Backbone=LLaMA3-8B2026.06 | 94.12 | 94.24 | 92.96 | 97.58 | 97.44 | 97.42 | 95.63 | — | |
| TIESSpace=CoreSpace, Backbone=LLaMA3-8B2026.06 | 93.15 | 83.46 | 91.65 | 99.19 | 93.46 | 97.71 | 93.1 | 93.4 | |
| LoRA-LEGOBackbone=LLaMA3-8B2026.06 | 92.84 | 58.51 | 87.25 | 99.19 | 87.47 | 95.81 | 86.84 | — | |
| GeoMergeBase Merging Algorithm=Task Arithmetic, Backbone=Llama-3 8B2026.04 | 92.159 | 83.749 | 91.416 | 100 | 90.274 | 93.665 | 91.877 | — | |
| RobustMergeBackbone=LLaMA3-8B2026.06 | 92.13 | 60.98 | 89.14 | 100 | 89.56 | 96.2 | 88 | — | |
| DARE-TIESSpace=CoreSpace, Backbone=LLaMA3-8B2026.06 | 91.82 | 80.79 | 91.9 | 100 | 95.39 | 97.37 | 92.88 | 93.12 | |
| FinetunedBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Number of Merged Models=6, Metric Normalization Protocol=Absolute accuracies (%)2026.03 | 90.8 | 95.3 | 92.1 | 84.8 | 91.3 | 87.3 | 90.2 | — | |
| LoRA's absolute accuracyBackbone=LLaMA3-8B2026.06 | 90.31 | 94.49 | 92.5 | 89.86 | 91.58 | 96.52 | — | — | |
| CtMSpace=CtM, Merging Rule=TIES, Backbone=LLaMA3-8B2026.06 | 90.26 | 89.71 | 91.08 | 100.81 | 96.11 | 96.93 | 94.15 | — | |
| Iso-CSpace=KnOTS, Backbone=LLaMA3-8B2026.06 | 90.07 | 56.48 | 86.75 | 100 | 89.12 | 95.57 | 86.33 | 91.99 | |
| Iso-CSpace=Full, Backbone=LLaMA3-8B2026.06 | 90.06 | 67.6 | 90.91 | 100.81 | 83.51 | 96.64 | 88.25 | 91.38 | |
| Iso-CSpace=CoreSpace, Backbone=LLaMA3-8B2026.06 | 90.03 | 67.57 | 90.9 | 100.81 | 83.49 | 96.69 | 88.25 | 90.38 | |
| CtMSpace=CtM, Merging Rule=Iso-C, Backbone=LLaMA3-8B2026.06 | 89.72 | 67.67 | 90.73 | 100.81 | 88.07 | 96 | 88.83 | — | |
| Iso-CTSBackbone=LLaMA3-8B2026.06 | 87.83 | 68.76 | 90.91 | 100.81 | 85.02 | 95.96 | 88.21 | — | |
| KnOTS-DARE-TIESBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=LoRA-aware Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 76.4 | 88.2 | 39.9 | 99.2 | 79.6 | 75.6 | 76.5 | — | |
| EMR-MergingBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=Vanilla Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 75.3 | 87.5 | 41.2 | 88 | 70.3 | 73.9 | 72.7 | — | |
| SVDBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=LoRA-aware Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 67.7 | 87.2 | 41.7 | 95.7 | 77.6 | 76.6 | 74.4 | — | |
| TABackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=Vanilla Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 67.3 | 87.3 | 41.8 | 95.7 | 77.9 | 76.9 | 74.6 | — | |
| RobustMergeBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=LoRA-aware Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 66.4 | 87.2 | 41.8 | 94 | 77.7 | 77.3 | 74.1 | — | |
| FR-MergingBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=Vanilla Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 65.7 | 86.9 | 41.9 | 93.2 | 78.3 | 77.6 | 73.9 | — | |
| LinearBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=LoRA-aware Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 63.1 | 86.2 | 39.5 | 90.6 | 77.5 | 74.5 | 71.9 | — | |
| LoRA-LEGOBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=LoRA-aware Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 62.5 | 84.8 | 41.4 | 100 | 87 | 83.7 | 76.6 | — | |
| TIESBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=Vanilla Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 61.3 | 91.5 | 38.2 | 85.5 | 76.8 | 70.5 | 70.6 | — | |
| TARA-Variant ABackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=LoRA-aware Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 51.7 | 92.6 | 41.4 | 102.6 | 95.3 | 94.4 | 79.7 | — | |
| AdaMergingBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=Vanilla Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 47.5 | 92.9 | 41.3 | 102.6 | 93.8 | 94.2 | 78.7 | — | |
| TARA-Variant BBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=LoRA-aware Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 46.8 | 94.1 | 41.4 | 103.4 | 98.1 | 97.8 | 80.3 | — | |
| DARE-TIESBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=Vanilla Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 42 | 72.5 | 44.1 | 77.8 | 76.9 | 86.8 | 66.7 | — | |
| KnOTS-TIESBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=LoRA-aware Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 41.1 | 83.4 | 56.6 | 87.2 | 87.9 | 94.8 | 75.2 | — | |
| Iso-CTSBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=Vanilla Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 37.1 | 57.1 | 39.8 | 59.4 | 49.8 | 95.3 | 56.4 | — | |
| Iso-CBackbone=LLaMA-3 8B, Fine-tuning Method (LoRA)=True, Merging Category=Vanilla Merging, Number of Merged Models=6, Metric Normalization Protocol=Normalized to finetuned (%)2026.03 | 35 | 57.8 | 40.5 | 64.1 | 51.1 | 100.2 | 58.1 | — |