Natural Language Inference on MNLI (Accuracy)
86.2AccuracyStandard MHA
Evaluation Results
| Method | Links | |
|---|---|---|
| Standard MHAMemory (GB)=15.22025.12 | 86.2 | |
| MAHAMemory (GB)=6.72025.12 | 86 | |
| BigBirdMemory (GB)=10.32025.12 | 85.9 | |
| LongformerMemory (GB)=9.12025.12 | 85.7 | |
| PerformerMemory (GB)=8.52025.12 | 85.1 | |
| ReformerMemory (GB)=7.82025.12 | 84.3 | |
| GPT-42025.12 | 80.1 | |
| CLGModel=Qwen2.5-72B, Shots=1282025.06 | 77.29 | |
| CLGShot count=128-shot, Model=Qwen2.5-72B2025.06 | 77.29 | |
| CLGModel=Llama3-70B, Shots=1282025.06 | 76.37 | |
| Best-of-NModel=Llama3-70B, Shots=1282025.06 | 70.92 | |
| Latent-BayesianModel=Llama3-70B, Shots=1282025.06 | 69.84 | |
| C2POBackbone=DeepSeek2025.12 | 65.9 | |
| BGE-KMeansModel=Llama3-70B, Shots=1282025.06 | 65.09 | |
| UniBiasModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 64.65 | |
| CCModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 63.89 | |
| ICLModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 62.43 | |
| DCModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 61.86 | |
| RandomModel=Llama3-70B, Shots=1282025.06 | 61.83 | |
| RandomShot count=128-shot, Model=Qwen2.5-72B2025.06 | 61.57 | |
| Best-of-NModel=Qwen2.5-72B, Shots=1282025.06 | 60.57 | |
| CLGShot count=1024-shot, Model=Qwen2.5-72B2025.06 | 58.94 | |
| EPR-KMeansModel=Qwen2.5-72B, Shots=1282025.06 | 58.41 | |
| RandomModel=Qwen2.5-72B, Shots=1282025.06 | 58.2 | |
| PCModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 57.47 | |
| RandomShot count=1024-shot, Model=Qwen2.5-72B2025.06 | 55.08 | |
| UniBiasModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 54.97 | |
| ICLModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 53.83 | |
| CCModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 53.36 | |
| DCModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 52.19 | |
| BM25-MajorModel=Llama3-70B, Shots=1282025.06 | 51.36 | |
| EPR-KMeansModel=Llama3-70B, Shots=1282025.06 | 49.78 | |
| BGE-KMeansModel=Qwen2.5-72B, Shots=1282025.06 | 49.65 | |
| Latent-BayesianModel=Qwen2.5-72B, Shots=1282025.06 | 47.96 | |
| BM25-MajorModel=Qwen2.5-72B, Shots=1282025.06 | 45.94 | |
| PCModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 45.38 |