List your company
ResearchFormerly WizWand

Natural Language Inference on MNLI (Accuracy)

86.2AccuracyStandard MHA
43.747254.768665.7976.8114May 31, 2024Sep 4, 2024Dec 9, 2024Mar 15, 2025Jun 19, 2025Sep 23, 2025Dec 29, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Standard MHAMemory (GB)=15.22025.1286.2
MAHAMemory (GB)=6.72025.1286
BigBirdMemory (GB)=10.32025.1285.9
LongformerMemory (GB)=9.12025.1285.7
PerformerMemory (GB)=8.52025.1285.1
ReformerMemory (GB)=7.82025.1284.3
GPT-42025.1280.1
CLGModel=Qwen2.5-72B, Shots=1282025.0677.29
CLGShot count=128-shot, Model=Qwen2.5-72B2025.0677.29
CLGModel=Llama3-70B, Shots=1282025.0676.37
Best-of-NModel=Llama3-70B, Shots=1282025.0670.92
Latent-BayesianModel=Llama3-70B, Shots=1282025.0669.84
C2POBackbone=DeepSeek2025.1265.9
BGE-KMeansModel=Llama3-70B, Shots=1282025.0665.09
UniBiasModel=Llama-2 13b, ICL Setting=1-shot2024.0564.65
CCModel=Llama-2 13b, ICL Setting=1-shot2024.0563.89
ICLModel=Llama-2 13b, ICL Setting=1-shot2024.0562.43
DCModel=Llama-2 13b, ICL Setting=1-shot2024.0561.86
RandomModel=Llama3-70B, Shots=1282025.0661.83
RandomShot count=128-shot, Model=Qwen2.5-72B2025.0661.57
Best-of-NModel=Qwen2.5-72B, Shots=1282025.0660.57
CLGShot count=1024-shot, Model=Qwen2.5-72B2025.0658.94
EPR-KMeansModel=Qwen2.5-72B, Shots=1282025.0658.41
RandomModel=Qwen2.5-72B, Shots=1282025.0658.2
PCModel=Llama-2 13b, ICL Setting=1-shot2024.0557.47
RandomShot count=1024-shot, Model=Qwen2.5-72B2025.0655.08
UniBiasModel=Llama-2 7b, ICL Setting=1-shot2024.0554.97
ICLModel=Llama-2 7b, ICL Setting=1-shot2024.0553.83
CCModel=Llama-2 7b, ICL Setting=1-shot2024.0553.36
DCModel=Llama-2 7b, ICL Setting=1-shot2024.0552.19
BM25-MajorModel=Llama3-70B, Shots=1282025.0651.36
EPR-KMeansModel=Llama3-70B, Shots=1282025.0649.78
BGE-KMeansModel=Qwen2.5-72B, Shots=1282025.0649.65
Latent-BayesianModel=Qwen2.5-72B, Shots=1282025.0647.96
BM25-MajorModel=Qwen2.5-72B, Shots=1282025.0645.94
PCModel=Llama-2 7b, ICL Setting=1-shot2024.0545.38