Natural Language Inference on RTE
93.5AccuracyEnumerate
Evaluation Results
| Method | Links | |
|---|---|---|
| EnumerateModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.04 | 93.5 | |
| OriginalModel=GPT-oss, Scope=–2026.04 | 92.8 | |
| GRAPEModel=GPT-oss, Scope=Global, Number of experts pruned=2e2026.04 | 92.6 | |
| Count-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.04 | 92.4 | |
| DEKModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.04 | 92.4 | |
| EnumerateModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.04 | 92.1 | |
| CoreSpaceBackbone=Llama-3, Scaling factor (λ)=0.52026.04 | 92.03 | |
| GRAPEModel=GPT-oss, Scope=Global, Number of experts pruned=4e2026.04 | 91.9 | |
| Count-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.04 | 91.3 | |
| TSVBackbone=Llama-3, Scaling factor (λ)=0.62026.04 | 91.3 | |
| TA + ENMPBackbone=Llama-3, Scaling factor (λ)=0.3, ENMP (Elective Neuron Modification Protocol)=true2026.04 | 91.06 | |
| KnOTS + ENMPBackbone=Llama-3, Scaling factor (λ)=1.1, Top-k (%)=90, ENMP (Elective Neuron Modification Protocol)=true2026.04 | 91.06 | |
| CoreSpace + ENMPBackbone=Llama-3, Scaling factor (λ)=0.5, ENMP (Elective Neuron Modification Protocol)=true2026.04 | 91.06 | |
| Router-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.04 | 90.6 | |
| DEKModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.04 | 90.6 | |
| KnOTSBackbone=Llama-3, Scaling factor (λ)=1.1, Top-k (%)=902026.04 | 90.58 | |
| LATModel=microsoft/deberta-xxlarge-v2-mnli2023.10 | 90 | |
| Individual TaskBackbone=Llama-3, Evaluation Protocol=fine-tuned on single tasks2026.04 | 89.86 | |
| TIESBackbone=Llama-3, Scaling factor (λ)=1.2, Top-k (%)=802026.04 | 89.86 | |
| TSV + ENMPBackbone=Llama-3, Scaling factor (λ)=0.6, ENMP (Elective Neuron Modification Protocol)=true2026.04 | 89.86 | |
| SKiC(Skills discoverd by LLM)Model=ChatGPT, Skill Discovery=Automatic discovery by LLM2023.08 | 89.8 | |
| Router-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.04 | 89.5 | |
| TIES + ENMPBackbone=Llama-3, Scaling factor (λ)=1.2, Top-k (%)=80, ENMP (Elective Neuron Modification Protocol)=true2026.04 | 89.37 | |
| TABackbone=Llama-3, Scaling factor (λ)=0.32026.04 | 89.13 | |
| DARE + ENMPBackbone=Llama-3, Scaling factor (λ)=1.1, Top-k (%)=80, Drop rate (p)=0.1, ENMP (Elective Neuron Modification Protocol)=true2026.04 | 88.65 | |
| Traditional MTL2024.05 | 88.6 | |
| DAREBackbone=Llama-3, Scaling factor (λ)=1.1, Top-k (%)=80, Drop rate (p)=0.12026.04 | 87.68 | |
| CALDERABackbone=LLaMa-3 8B, Rank=64, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.42024.05 | 87 | |
| CALDERABackbone=LLaMa-3 8B, Rank=128, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.52024.05 | 86.64 | |
| Self-consistencyPrompting strategy=Self-consistency, Backbone model=PaLM-540B2022.03 | 86.3 | |
| CALDERABackbone=LLaMa-2 7B, Rank=256, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.72024.05 | 86.28 | |
| CALDERABackbone=LLaMa-3 8B, Rank=64, BQ=2, BL (= BR)=16, RHT FT=Yes, Avg Bits=2.42024.05 | 86.28 | |
| CALDERABackbone=LLaMa-3 8B, Rank=256, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.72024.05 | 86.28 | |
| CALDERABackbone=LLaMa-2 7B, Rank=64, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.42024.05 | 85.56 | |
| CALDERABackbone=LLaMa-3 8B, Rank=64, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.42024.05 | 85.56 | |
| ICLBackbone=Llama-2 70B, Shots=128-shots2024.05 | 85.5 | |
| CALDERABackbone=LLaMa-3 8B, Rank=256, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.72024.05 | 85.2 | |
| COTModel=ChatGPT2023.08 | 85.2 | |
| CALDERABackbone=LLaMa-2 7B, Rank=64, BQ=2, BL (= BR)=16, RHT FT=Yes, Avg Bits=2.42024.05 | 85.19 | |
| CALDERABackbone=LLaMa-3 8B, Rank=128, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.52024.05 | 84.84 | |
| Standard-prompting (no-rationale)Prompting strategy=no-rationale, Backbone model=PaLM-540B2022.03 | 84.8 | |
| CALDERABackbone=LLaMa-3 8B, Rank=64, BQ=2, BL (= BR)=16, RHT FT=No, Avg Bits=2.42024.05 | 84.48 | |
| ColD-Fusion2022.12 | 84.48 | |
| CALDERABackbone=LLaMa-2 7B, Rank=128, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.52024.05 | 84.12 | |
| CALDERABackbone=LLaMa-2 7B, Rank=128, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.52024.05 | 83.75 | |
| CALDERABackbone=LLaMa-2 7B, Rank=256, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.72024.05 | 83.75 | |
| kNN-promptingBackbone=Llama-2 70B, Shots=128-shots2024.05 | 83.6 | |
| FADS-ICLBackbone=Llama-2 70B, Shots=128-shots2024.05 | 83.6 | |
| AdamWBackbone=RoBERTa-large, Optimization Method=AdamW, Finetuning Protocol=Few-shot, Iterations=10K2025.11 | 83.6 | |
| VanillaBackbone=Qwen2.5-7B-Instruct2026.05 | 83.5 | |
| Fisher MergingValidation=true2024.05 | 83.3 | |
| LoRABackbone=Qwen2.5-7B-Instruct2026.05 | 83.3 | |
| EVA (text)Backbone=Qwen2.5-7B-Instruct2026.05 | 83.3 | |
| SafeDecodingBackbone=Qwen2.5-7B-Instruct2026.05 | 83.1 | |
| Individual2024.05 | 82.7 | |
| CALDERABackbone=LLaMa-2 7B, Rank=64, BQ=2, BL (= BR)=16, RHT FT=No, Avg Bits=2.42024.05 | 82.31 | |
| Qwen3-30B-A3BPruning ratio=0%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 82.31 | |
| Qwen3-30B-A3BPruning ratio=0%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 82.31 | |
| Multitask2022.12 | 82.17 | |
| OPT-IML 175BEvaluation protocol=few-shot, k=52022.12 | 82.1 | |
| CCSModel=microsoft/deberta-xxlarge-v2-mnli2023.10 | 82 | |
| FLAPPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 81.95 | |
| PaLM 2-Mprompting=1-shot2023.05 | 81.9 | |
| PaLM-2 Mprotocol=1-shot2023.11 | 81.8 | |
| EMR-MERGINGValidation=false2024.05 | 81.8 | |
| ICLBackbone=Llama-1 30B, Shots=128-shots2024.05 | 81.5 | |
| FADS-ICLNumber of training shots (m)=32, LLM scale=1.5B2024.05 | 81.4 | |
| Circuit BreakersBackbone=Qwen2.5-7B-Instruct2026.05 | 81.3 | |
| CALDERABackbone=LLaMa-2 7B, Rank=64, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.42024.05 | 81.23 | |
| RegMeanValidation=true2024.05 | 81.2 | |
| Weight AveragingValidation=false2024.05 | 81.2 | |
| Ties-MergingValidation=false2024.05 | 81.2 | |
| FADS-ICLBackbone=Llama-2 13B, Shots=128-shots2024.05 | 81.1 | |
| FADS-ICLBackbone=Llama-1 30B, Shots=128-shots2024.05 | 81.1 | |
| OPT-IML 175BEvaluation protocol=0-shot2022.12 | 80.9 | |
| RoBERTa (Gao et al., 2021)# Params=1.0x, Evaluation Protocol=Full fine-tuning2022.12 | 80.9 | |
| MC-SMoEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 80.87 | |
| T0-11BParameters=11B2023.02 | 80.83 | |
| LEDBackbone=Qwen2.5-7B-Instruct2026.05 | 80.7 | |
| kNN-promptBackbone=Llama-2 70B, Shots=128-shots2024.05 | 80.2 | |
| Falcon-180Bprotocol=1-shot2023.11 | 80.1 | |
| FLAN 137BEvaluation protocol=few-shot, k=variable, Training strategy=leave-one-category-out2022.12 | 79.9 | |
| Fine-tuningn=256, Update model parameters=true2022.12 | 79.8 | |
| RSPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 79.78 | |
| PaLM 2-Lprompting=1-shot2023.05 | 79.3 | |
| PaLM-2 Lprotocol=1-shot2023.11 | 79.3 | |
| CoT-promptingPrompting strategy=Chain-of-Thought, Backbone model=PaLM-540B2022.03 | 79.1 | |
| PaLMprompting=1-shot2023.05 | 78.7 | |
| PaLM 2-Sprompting=1-shot2023.05 | 78.7 | |
| PaLMprotocol=1-shot2023.11 | 78.7 | |
| PaLM-2 Sprotocol=1-shot2023.11 | 78.7 | |
| LOZOBackbone=RoBERTa-Large, Evaluation Protocol=prompt-conditioned, Time Constraint=equal wall-clock time2025.11 | 78.7 | |
| MC-SMoEPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 78.34 | |
| FLAN 137BEvaluation protocol=0-shot, Training strategy=leave-one-category-out2022.12 | 78.3 | |
| UniBiasModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 78.23 | |
| Ties-MergingValidation=true2024.05 | 78 | |
| MoNEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 77.98 | |
| FLAPPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.07 | 77.62 | |
| FoTBackbone=RoBERTa_LARGE, Samples per class=16, Pre-trained prompt=true2025.06 | 77.62 | |
| Prompt TuningBackbone=RoBERTa_LARGE, Samples per class=16, Pre-trained prompt=true2025.06 | 77.13 |