Commonsense Reasoning on XCOPA
74.5AccuracySailor2
Evaluation results
| Method | Links | |
|---|---|---|
| Sailor2Param=8B2026.02 | 74.5 | |
| SeaLLM v3Param=7B2026.02 | 70.53 | |
| SEA-LION v3.5Param=8B2026.02 | 70.37 | |
| MultilingualParam=7B2026.02 | 70.3 | |
| Parallel Only (OPENSEAL)Param=7B2026.02 | 70.2 | |
| XGLM7.5Bk-shot=162022.04 | 62.5 | |
| XGLM7.5Bk-shot=42022.04 | 61.4 | |
| BYOL-mriParameters=12B, Training Stage=CPT2026.01 | 61.2 | |
| 1B Model (Multilingual classifier)Filtering Strategy=Multilingual (ML), Model Scale=1B2026.04 | 60.8 | |
| XGLM7.5Bk-shot=12022.04 | 60.6 | |
| BYOL-mriParameters=4B, Training Stage=CPT2026.01 | 60.6 | |
| Gemma 2Number of Parameters=1B, Evaluation Protocol=Zero-shot2024.08 | 59.7 | |
| 1B Model (HQ filtering)Filtering Strategy=Monolingual high-quality (HQ), Model Scale=1B2026.04 | 59.2 | |
| 1B Model (No filtering)Filtering Strategy=No filtering, Model Scale=1B2026.04 | 58.6 | |
| mGPT13Bk-shot=42022.04 | 58.3 | |
| mGPT13Bk-shot=02022.04 | 58.2 | |
| mGPT13Bk-shot=12022.04 | 57.9 | |
| XGLMNumber of Parameters=1.7B, Evaluation Protocol=Zero-shot2024.08 | 57.6 | |
| mGPT13Bk-shot=162022.04 | 57.3 | |
| BYOL-mriParameters=1B, Training Stage=CPT2026.01 | 57 | |
| XGLM1.7Bk-shot=12022.04 | 56.8 | |
| XGLM1.7Bk-shot=42022.04 | 56.2 | |
| XGLM1.7Bk-shot=162022.04 | 56.1 | |
| LLaMA 3Number of Parameters=1B, Evaluation Protocol=Zero-shot2024.08 | 55.9 | |
| mGPT1.3Bk-shot=02022.04 | 55.5 | |
| XGLM1.7Bk-shot=02022.04 | 55.5 | |
| XGLM7.5Bk-shot=02022.04 | 55.5 | |
| XGLMNumber of Parameters=564M, Evaluation Protocol=Zero-shot2024.08 | 55.2 | |
| GoldfishNumber of Parameters=124M, Evaluation Protocol=Zero-shot2024.08 | 55.1 | |
| mGPT1.3Bk-shot=12022.04 | 54.9 | |
| mGPT1.3Bk-shot=42022.04 | 54.8 | |
| mGPT1.3Bk-shot=162022.04 | 54.8 | |
| Gemma 2Number of Parameters=270M, Evaluation Protocol=Zero-shot2024.08 | 54.6 | |
| BLOOMNumber of Parameters=560M, Evaluation Protocol=Zero-shot2024.08 | 54.4 | |
| Chance2024.08 | 50 |