Natural Language Inference on XNLI
87.1AccuracymT5
Evaluation results
| Method | Links | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| mT5Scale=XXL, Evaluation Protocol=Translate-train2021.05 | 87.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=XXL, Evaluation Protocol=Translate-train2021.05 | 85.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=XL, Evaluation Protocol=Translate-train2021.05 | 85.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLARETransfer Strategy=Translate-Train, Base Model=Gemma 2 9B2025.01 | 85.01 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=XL, Evaluation Protocol=Translate-train2021.05 | 85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=XXL, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 85 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRATransfer Strategy=Translate-Train, Base Model=Gemma 2 9B2025.01 | 84.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLARE MTTransfer Strategy=Translate-Train, Base Model=Gemma 2 9B2025.01 | 84.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| X-MixupTransfer Strategy=Translate-Train, Base Model=Gemma 2 9B2025.01 | 84.62 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=Large, Evaluation Protocol=Translate-train2021.05 | 84.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=XXL, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 83.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=XL, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 82.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=Large, Evaluation Protocol=Translate-train2021.05 | 82.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=XL, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 82.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLARE MTTransfer Strategy=Translate-Train, Base Model=XLM-R Large2025.01 | 81.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=Large, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 81.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLARETransfer Strategy=Translate-Train, Base Model=mT5-XL2025.01 | 81 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLARETransfer Strategy=Translate-Train, Base Model=XLM-R Large2025.01 | 80.99 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLARETransfer Strategy=Translate-Train, Base Model=Llama 3.1 8B2025.01 | 80.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLARE MTTransfer Strategy=Translate-Train, Base Model=Llama 3.1 8B2025.01 | 80.83 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FLARE MTTransfer Strategy=Translate-Train, Base Model=mT5-XL2025.01 | 80.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRATransfer Strategy=Translate-Train, Base Model=Llama 3.1 8B2025.01 | 80.74 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| input-level fusionTransfer Strategy=Translate-Train, Base Model=Llama 3.1 8B2025.01 | 80.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| input-level fusionTransfer Strategy=Translate-Train, Base Model=Gemma 2 9B2025.01 | 80.53 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=Base, Evaluation Protocol=Translate-train2021.05 | 80.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRATransfer Strategy=Translate-Train, Base Model=XLM-R Large2025.01 | 80.49 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma 2 9BTransfer Strategy=Zero-Shot Cross-Lingual Transfer2025.01 | 80.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| X-MixupTransfer Strategy=Translate-Train, Base Model=Llama 3.1 8B2025.01 | 80.22 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Gemma 2 9BTransfer Strategy=Translate-Test2025.01 | 79.99 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=Base, Evaluation Protocol=Translate-train2021.05 | 79.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LoRATransfer Strategy=Translate-Train, Base Model=mT5-XL2025.01 | 79.79 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=Large, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 79.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| X-MixupTransfer Strategy=Translate-Train, Base Model=mT5-XL2025.01 | 79.63 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| X-MixupTransfer Strategy=Translate-Train, Base Model=XLM-R Large2025.01 | 79.47 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama 3.1 8BTransfer Strategy=Translate-Test2025.01 | 79.08 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5-XLTransfer Strategy=Translate-Test2025.01 | 79.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| input-level fusionTransfer Strategy=Translate-Train, Base Model=mT5-XL2025.01 | 78.81 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5-XLTransfer Strategy=Zero-Shot Cross-Lingual Transfer2025.01 | 77.92 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Llama 3.1 8BTransfer Strategy=Zero-Shot Cross-Lingual Transfer2025.01 | 77.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| input-level fusionTransfer Strategy=Translate-Train, Base Model=XLM-R Large2025.01 | 77.24 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XLM-R LargeTransfer Strategy=Translate-Test2025.01 | 77.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XLM-R LargeTransfer Strategy=Zero-Shot Cross-Lingual Transfer2025.01 | 76.95 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=Small, Evaluation Protocol=Translate-train2021.05 | 76.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=Base, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 75.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ByT5Scale=Base, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 75.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=Small, Evaluation Protocol=Translate-train2021.05 | 75.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Few-shotModel=Qwen3-8B, Training Size=1002026.02 | 70.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.03 | — | |
| ByT5Scale=Small, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 69.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA3-8BVariant=Baseline2026.01 | 69 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA3-8BVariant=LoRA-Drop, drop-ratio (rho)=0.5, temporal window (k)=32026.01 | 68.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mT5Scale=Small, Evaluation Protocol=Cross-lingual zero-shot transfer2021.05 | 67.5 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-7BVariant=Baseline2026.01 | 67 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen2.5-7BVariant=LoRA-Drop, drop-ratio (rho)=0.5, temporal window (k)=32026.01 | 66.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA2-7BVariant=Baseline2026.01 | 64 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LLaMA2-7BVariant=LoRA-Drop, drop-ratio (rho)=0.5, temporal window (k)=32026.01 | 63.9 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| QwenMethod Variant=base2026.01 | 59.93 | 60.45 | 60.75 | 66.22 | 58.02 | 65.56 | 64.47 | 55.56 | 61.87 | 40.31 | 57.8 | 59.18 | 51.07 | 62.11 | 63.81 | 71.73 | 6.78 | — | |
| CC-TUNING+SDRRLModel Backbone=Qwen2.5-7B2025.06 | 53.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ML-SFT+SDRRLModel Backbone=Qwen2.5-7B2025.06 | 52.36 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CC-TUNINGModel Backbone=Qwen2.5-7B2025.06 | 51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ML-SFT+ENModel Backbone=Qwen2.5-7B2025.06 | 50.76 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CC-TUNING+ENModel Backbone=Qwen2.5-7B2025.06 | 49.48 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ROASTModel=Gemma3-1B, Training Size=1000, Aggregation=no-group2026.02 | 48.84 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.01 | — | |
| ML-SFTModel Backbone=Qwen2.5-7B2025.06 | 48.23 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CC-TUNING+MTModel Backbone=Qwen2.5-7B2025.06 | 48.13 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ML-SFT+MTModel Backbone=Qwen2.5-7B2025.06 | 47.05 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ROASTModel=Qwen3-0.6B, Training Size=100, Aggregation=no-group2026.02 | 46.26 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 0.02 | — | |
| SEA-LION v3.5Param=8B2026.02 | 46.11 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MultilingualParam=7B2026.02 | 45.73 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Parallel Only (OPENSEAL)Param=7B2026.02 | 45.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Apertus-v1.1-4Bmode=Zero-shot, model_type=Base2026.05 | 45.03 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XGLM7.5Bk-shot=02022.04 | 44.7 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Sailor2Param=8B2026.02 | 44.59 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Apertus-8Bmode=Zero-shot, model_type=Base2026.05 | 44.09 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SeaLLM v3Param=7B2026.02 | 43.28 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| LlamaMethod Variant=base2026.01 | 43.1 | 38.88 | 42.28 | 43.65 | 40.58 | 46.95 | 44.53 | 41.78 | 43.75 | 41.1 | 40.68 | 44.17 | 36.09 | 44.81 | 44.63 | 52.63 | 2.82 | — | |
| Qwen3-4B-Basemode=Zero-shot, model_type=Base2026.05 | 43 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Apertus-v1.1-1.5Bmode=Zero-shot, model_type=Base2026.05 | 42.94 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mGPT13Bk-shot=02022.04 | 42.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XGLM1.7Bk-shot=02022.04 | 42.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MLFiltering Strategy=ML2026.04 | 41.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| 1B Model (Multilingual classifier)Filtering Strategy=Multilingual (ML), Model Scale=1B2026.04 | 41.89 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-1.7B-Basemode=Zero-shot, model_type=Base2026.05 | 41.66 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Apertus-v1.1-0.5Bmode=Zero-shot, model_type=Base2026.05 | 41.51 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mGPT13Bk-shot=42022.04 | 41.4 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| EuroLLM-1.7Bmode=Zero-shot, model_type=Base2026.05 | 40.88 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| HQFiltering Strategy=HQ2026.04 | 40.72 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| 1B Model (HQ filtering)Filtering Strategy=Monolingual high-quality (HQ), Model Scale=1B2026.04 | 40.72 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| mGPT1.3Bk-shot=02022.04 | 40.6 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| MKC-eFiltering Strategy=MKC-e2026.04 | 40.56 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FTBase Model=LLaMA-7B, Training Context=Alpaca-En2023.11 | 40.3 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| SmolLM-3B-Basemode=Zero-shot, model_type=Base2026.05 | 40.28 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FTBase Model=LLaMA-7B, Training Context=Alpaca-X2023.11 | 40.2 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XGLM7.5Bk-shot=42022.04 | 40.1 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XGLM7.5Bk-shot=162022.04 | 40 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| ML (15%)Filtering Strategy=ML (15%)2026.04 | 39.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Qwen3-0.6B-Basemode=Zero-shot, model_type=Base2026.05 | 39.55 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| Parrot-7BBase Model=Parrot-7B2023.11 | 39 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| XGLM1.7Bk-shot=42022.04 | 38.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| FTBase Model=LLaMA-7B, Training Context=Bilingual2023.11 | 38.8 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | |
| CC-TUNINGModel Backbone=LLaMA-3.1-8B2025.06 | 38.42 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — | — |