List your company
ResearchFormerly WizWand

Causal Reasoning on XCOPA

94.4AccuracyPaLM 2
46.76859.13471.583.866May 17, 2023Nov 17, 2023May 19, 2024Nov 20, 2024May 23, 2025Nov 23, 2025May 27, 2026
Updated 5mo ago

Evaluation results

MethodLinks
PaLM 2Number of exemplars (k-shot)=4, Model variant=Original2023.0594.4
PaLM + CoTNumber of exemplars (k-shot)=4, Chain-of-thought prompting (CoT)=true2023.0589.9
BYOL-nyaParameters=12B, Pre-training Type=CPT2026.0171.2
BYOL-nyaParameters=4B, Pre-training Type=CPT2026.0170
Apertus-8Bmode=Zero-shot, model_type=Base2026.0565.69
QuaSARBackbone=Llama-3-8B2025.0265
Apertus-v1.1-4Bmode=Zero-shot, model_type=Base2026.0563.82
HQ seedFiltering Strategy=HQ seed2026.0462
Qwen3-4B-Basemode=Zero-shot, model_type=Base2026.0561.82
ML (15%)Filtering Strategy=ML (15%)2026.0461.6
BYOL-nyaParameters=1B, Pre-training Type=CPT2026.0161.2
Gemma-3Parameters=12B, Pre-training Type=PT2026.0160.8
MLFiltering Strategy=ML2026.0460.8
ML (Q3)Filtering Strategy=ML (Q3)2026.0460.2
Apertus-v1.1-1.5Bmode=Zero-shot, model_type=Base2026.0559.76
MKC-eFiltering Strategy=MKC-e2026.0459.4
HQFiltering Strategy=HQ2026.0459.2
No filteringFiltering Strategy=No filtering2026.0458.6
Qwen3-1.7B-Basemode=Zero-shot, model_type=Base2026.0558.35
SmolLM-3B-Basemode=Zero-shot, model_type=Base2026.0558.02
Gemma-3Parameters=4B, Pre-training Type=PT2026.0157.2
CoTBackbone=Llama-3-8B2025.0256.9
baselineBackbone=Llama-3-8B2025.0256.4
DMoEModel Scale=1.7B, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Zero-shot2025.0656
ACROS2026.0555.8
EuroLLM-1.7Bmode=Zero-shot, model_type=Base2026.0555.76
DMoEModel Scale=1.7B, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Few-shot (4-shot)2025.0655.7
Branch-Train-MixModel Scale=1.7B, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Zero-shot2025.0655.6
Branch-Train-MixModel Scale=1.7B, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Few-shot (4-shot)2025.0655.5
Apertus-v1.1-0.5Bmode=Zero-shot, model_type=Base2026.0555.49
BLOOM + Continued Pre-trainingModel Scale=1.7B, Training Strategy=Continued Pre-training, Evaluation Protocol=Few-shot (4-shot)2025.0655.3
BLOOMModel Scale=1.7B, Training Strategy=Base, Evaluation Protocol=Few-shot (4-shot)2025.0655.2
BLOOMModel Scale=1.7B, Training Strategy=Base, Evaluation Protocol=Zero-shot2025.0655.1
BLOOM + Continued Pre-trainingModel Scale=1.7B, Training Strategy=Continued Pre-training, Evaluation Protocol=Zero-shot2025.0655
Qwen3-0.6B-Basemode=Zero-shot, model_type=Base2026.0554.96
GemmaParameters=270M2026.0554.8
DMoEModel Scale=560M, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Few-shot (4-shot)2025.0654.7
DMoEModel Scale=560M, Training Strategy=Dynamic Mixture-of-Experts, Evaluation Protocol=Zero-shot2025.0654.4
Branch-Train-MixModel Scale=560M, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Zero-shot2025.0654.1
BLOOMModel Scale=560M, Training Strategy=Base, Evaluation Protocol=Zero-shot2025.0653.9
Branch-Train-MixModel Scale=560M, Training Strategy=Branch-Train-Mix, Evaluation Protocol=Few-shot (4-shot)2025.0653.9
BLOOM + Continued Pre-trainingModel Scale=560M, Training Strategy=Continued Pre-training, Evaluation Protocol=Few-shot (4-shot)2025.0653.8
BLOOM + Continued Pre-trainingModel Scale=560M, Training Strategy=Continued Pre-training, Evaluation Protocol=Zero-shot2025.0653.6
SmolLM2-1.7Bmode=Zero-shot, model_type=Base2026.0553.51
BLOOMModel Scale=560M, Training Strategy=Base, Evaluation Protocol=Few-shot (4-shot)2025.0653.4
ApertusParameters=8B, Pre-training Type=25092026.0153.2
Qwen2Parameters=0.5B2026.0553.2
Frozen BP2026.0553
Qwen-3Parameters=8B, Pre-training Type=Base2026.0152.2
Qwen-3Parameters=1.7B, Pre-training Type=Base2026.0152
SmolLM2Variant=Base2026.0552
Gemma-3Parameters=1B, Pre-training Type=PT2026.0151.8
Llama-3.1Parameters=8B2026.0151.2
Qwen-3Parameters=14B, Pre-training Type=Base2026.0151
Llama-3.2Parameters=1B2026.0148.6