List your company
ResearchFormerly WizWand

Natural Language Inference on RTE

93.5AccuracyEnumerate
76.475280.895185.31589.7349Mar 21, 2022Nov 28, 2022Aug 8, 2023Apr 16, 2024Dec 25, 2024Sep 3, 2025May 14, 2026
Updated 5mo ago

Evaluation results

MethodLinks
EnumerateModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.0493.5
OriginalModel=GPT-oss, Scope=–2026.0492.8
GRAPEModel=GPT-oss, Scope=Global, Number of experts pruned=2e2026.0492.6
Count-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.0492.4
DEKModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.0492.4
EnumerateModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.0492.1
CoreSpaceBackbone=Llama-3, Scaling factor (λ)=0.52026.0492.03
GRAPEModel=GPT-oss, Scope=Global, Number of experts pruned=4e2026.0491.9
Count-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.0491.3
TSVBackbone=Llama-3, Scaling factor (λ)=0.62026.0491.3
TA + ENMPBackbone=Llama-3, Scaling factor (λ)=0.3, ENMP (Elective Neuron Modification Protocol)=true2026.0491.06
KnOTS + ENMPBackbone=Llama-3, Scaling factor (λ)=1.1, Top-k (%)=90, ENMP (Elective Neuron Modification Protocol)=true2026.0491.06
CoreSpace + ENMPBackbone=Llama-3, Scaling factor (λ)=0.5, ENMP (Elective Neuron Modification Protocol)=true2026.0491.06
Router-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=2e2026.0490.6
DEKModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.0490.6
KnOTSBackbone=Llama-3, Scaling factor (λ)=1.1, Top-k (%)=902026.0490.58
LATModel=microsoft/deberta-xxlarge-v2-mnli2023.1090
Individual TaskBackbone=Llama-3, Evaluation Protocol=fine-tuned on single tasks2026.0489.86
TIESBackbone=Llama-3, Scaling factor (λ)=1.2, Top-k (%)=802026.0489.86
TSV + ENMPBackbone=Llama-3, Scaling factor (λ)=0.6, ENMP (Elective Neuron Modification Protocol)=true2026.0489.86
SKiC(Skills discoverd by LLM)Model=ChatGPT, Skill Discovery=Automatic discovery by LLM2023.0889.8
Router-guidedModel=GPT-oss, Scope=Local, Number of experts pruned=4e2026.0489.5
TIES + ENMPBackbone=Llama-3, Scaling factor (λ)=1.2, Top-k (%)=80, ENMP (Elective Neuron Modification Protocol)=true2026.0489.37
TABackbone=Llama-3, Scaling factor (λ)=0.32026.0489.13
DARE + ENMPBackbone=Llama-3, Scaling factor (λ)=1.1, Top-k (%)=80, Drop rate (p)=0.1, ENMP (Elective Neuron Modification Protocol)=true2026.0488.65
Traditional MTL2024.0588.6
DAREBackbone=Llama-3, Scaling factor (λ)=1.1, Top-k (%)=80, Drop rate (p)=0.12026.0487.68
CALDERABackbone=LLaMa-3 8B, Rank=64, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.42024.0587
CALDERABackbone=LLaMa-3 8B, Rank=128, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.52024.0586.64
Self-consistencyPrompting strategy=Self-consistency, Backbone model=PaLM-540B2022.0386.3
CALDERABackbone=LLaMa-2 7B, Rank=256, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.72024.0586.28
CALDERABackbone=LLaMa-3 8B, Rank=64, BQ=2, BL (= BR)=16, RHT FT=Yes, Avg Bits=2.42024.0586.28
CALDERABackbone=LLaMa-3 8B, Rank=256, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.72024.0586.28
CALDERABackbone=LLaMa-2 7B, Rank=64, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.42024.0585.56
CALDERABackbone=LLaMa-3 8B, Rank=64, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.42024.0585.56
ICLBackbone=Llama-2 70B, Shots=128-shots2024.0585.5
CALDERABackbone=LLaMa-3 8B, Rank=256, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.72024.0585.2
COTModel=ChatGPT2023.0885.2
CALDERABackbone=LLaMa-2 7B, Rank=64, BQ=2, BL (= BR)=16, RHT FT=Yes, Avg Bits=2.42024.0585.19
CALDERABackbone=LLaMa-3 8B, Rank=128, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.52024.0584.84
Standard-prompting (no-rationale)Prompting strategy=no-rationale, Backbone model=PaLM-540B2022.0384.8
CALDERABackbone=LLaMa-3 8B, Rank=64, BQ=2, BL (= BR)=16, RHT FT=No, Avg Bits=2.42024.0584.48
ColD-Fusion2022.1284.48
CALDERABackbone=LLaMa-2 7B, Rank=128, BQ=2, BL (= BR)=4, RHT FT=Yes, Avg Bits=2.52024.0584.12
CALDERABackbone=LLaMa-2 7B, Rank=128, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.52024.0583.75
CALDERABackbone=LLaMa-2 7B, Rank=256, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.72024.0583.75
kNN-promptingBackbone=Llama-2 70B, Shots=128-shots2024.0583.6
FADS-ICLBackbone=Llama-2 70B, Shots=128-shots2024.0583.6
AdamWBackbone=RoBERTa-large, Optimization Method=AdamW, Finetuning Protocol=Few-shot, Iterations=10K2025.1183.6
VanillaBackbone=Qwen2.5-7B-Instruct2026.0583.5
Fisher MergingValidation=true2024.0583.3
LoRABackbone=Qwen2.5-7B-Instruct2026.0583.3
EVA (text)Backbone=Qwen2.5-7B-Instruct2026.0583.3
SafeDecodingBackbone=Qwen2.5-7B-Instruct2026.0583.1
Individual2024.0582.7
CALDERABackbone=LLaMa-2 7B, Rank=64, BQ=2, BL (= BR)=16, RHT FT=No, Avg Bits=2.42024.0582.31
Qwen3-30B-A3BPruning ratio=0%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0782.31
Qwen3-30B-A3BPruning ratio=0%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0782.31
Multitask2022.1282.17
OPT-IML 175BEvaluation protocol=few-shot, k=52022.1282.1
CCSModel=microsoft/deberta-xxlarge-v2-mnli2023.1082
FLAPPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0781.95
PaLM 2-Mprompting=1-shot2023.0581.9
PaLM-2 Mprotocol=1-shot2023.1181.8
EMR-MERGINGValidation=false2024.0581.8
ICLBackbone=Llama-1 30B, Shots=128-shots2024.0581.5
FADS-ICLNumber of training shots (m)=32, LLM scale=1.5B2024.0581.4
Circuit BreakersBackbone=Qwen2.5-7B-Instruct2026.0581.3
CALDERABackbone=LLaMa-2 7B, Rank=64, BQ=2, BL (= BR)=4, RHT FT=No, Avg Bits=2.42024.0581.23
RegMeanValidation=true2024.0581.2
Weight AveragingValidation=false2024.0581.2
Ties-MergingValidation=false2024.0581.2
FADS-ICLBackbone=Llama-2 13B, Shots=128-shots2024.0581.1
FADS-ICLBackbone=Llama-1 30B, Shots=128-shots2024.0581.1
OPT-IML 175BEvaluation protocol=0-shot2022.1280.9
RoBERTa (Gao et al., 2021)# Params=1.0x, Evaluation Protocol=Full fine-tuning2022.1280.9
MC-SMoEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0780.87
T0-11BParameters=11B2023.0280.83
LEDBackbone=Qwen2.5-7B-Instruct2026.0580.7
kNN-promptBackbone=Llama-2 70B, Shots=128-shots2024.0580.2
Falcon-180Bprotocol=1-shot2023.1180.1
FLAN 137BEvaluation protocol=few-shot, k=variable, Training strategy=leave-one-category-out2022.1279.9
Fine-tuningn=256, Update model parameters=true2022.1279.8
RSPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0779.78
PaLM 2-Lprompting=1-shot2023.0579.3
PaLM-2 Lprotocol=1-shot2023.1179.3
CoT-promptingPrompting strategy=Chain-of-Thought, Backbone model=PaLM-540B2022.0379.1
PaLMprompting=1-shot2023.0578.7
PaLM 2-Sprompting=1-shot2023.0578.7
PaLMprotocol=1-shot2023.1178.7
PaLM-2 Sprotocol=1-shot2023.1178.7
LOZOBackbone=RoBERTa-Large, Evaluation Protocol=prompt-conditioned, Time Constraint=equal wall-clock time2025.1178.7
MC-SMoEPruning ratio=50%, Base Model=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0778.34
FLAN 137BEvaluation protocol=0-shot, Training strategy=leave-one-category-out2022.1278.3
UniBiasModel=Llama-2 13b, ICL Setting=1-shot2024.0578.23
Ties-MergingValidation=true2024.0578
MoNEPruning ratio=25%, Backbone=Qwen3-30B-A3B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0777.98
FLAPPruning ratio=25%, Backbone=Qwen2-57B-A14B, Evaluation protocol=Zero-shot, Calibration samples=100, Calibration dataset=Zyda22025.0777.62
FoTBackbone=RoBERTa_LARGE, Samples per class=16, Pre-trained prompt=true2025.0677.62
Prompt TuningBackbone=RoBERTa_LARGE, Samples per class=16, Pre-trained prompt=true2025.0677.13