Causal Reasoning on COPA
95AccuracyFew-shot*
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Few-shot*Backbone=Qwen2.5-7B, Few-shot settings=3-shot OOD2025.09 | 95 | — | — | |
| ICRBackbone=Qwen2.5-7B2025.09 | 95 | — | — | |
| M²IVBackbone=Qwen2.5-7B2025.09 | 93.5 | — | — | |
| LIVEBackbone=Qwen2.5-7B2025.09 | 93.2 | — | — | |
| I2CLBackbone=Qwen2.5-7B2025.09 | 92.6 | — | — | |
| Zero-shotBackbone=Qwen2.5-7B2025.09 | 92 | — | — | |
| LATModel=microsoft/deberta-xxlarge-v2-mnli2023.10 | 90 | — | — | |
| GPT-NeoXEvaluation Protocol=1-shot2023.03 | 88 | — | — | |
| GPT-3Evaluation Protocol=1-shot2023.03 | 87 | — | — | |
| DenseBase Model=LLaMA-2-7B, Sparsity=Dense2025.06 | 87 | — | — | |
| MaskProBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 87 | — | — | |
| BloombergGPTEvaluation Protocol=1-shot2023.03 | 86 | — | — | |
| OPT-66BEvaluation Protocol=1-shot2023.03 | 86 | — | — | |
| SparsegptBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 86 | — | — | |
| BLOOM-176BEvaluation Protocol=1-shot2023.03 | 84 | — | — | |
| DenseBase Model=DeepSeek-7B, Sparsity=Dense2025.06 | 84 | — | — | |
| Pruner-ZBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 84 | — | — | |
| UniBiasModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 83.2 | — | — | |
| Few-shot*Backbone=Llama2-7B, Few-shot settings=3-shot OOD2025.09 | 82 | — | — | |
| SparsegptBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 81 | — | — | |
| FIXED_E=128Experts=1282026.05 | 81 | — | — | |
| EMO (Stage 5)E=64→1282026.05 | 80 | — | — | |
| Our Trained ModelModel Size=52B, Data type=FP16, Zero-shot=true2023.05 | 79.4 | — | — | |
| Our Trained ModelModel Size=52B, Data type=W8A8, Zero-shot=true2023.05 | 79.4 | — | — | |
| Pruner-ZBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 79 | — | — | |
| MaskProBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 79 | — | — | |
| Our Trained ModelModel Size=52B, Data type=W4, Zero-shot=true2023.05 | 78.2 | — | — | |
| PCModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 76.8 | — | — | |
| Our Trained ModelModel Size=13B, Data type=FP16, Zero-shot=true2023.05 | 76 | — | — | |
| Our Trained ModelModel Size=13B, Data type=W8A8, Zero-shot=true2023.05 | 76 | — | — | |
| FIXED_E=32Experts=322026.05 | 76 | — | — | |
| Our Trained ModelModel Size=6B, Data type=FP16, Zero-shot=true2023.05 | 75.4 | — | — | |
| CCModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 75.2 | — | — | |
| Our Trained ModelModel Size=6B, Data type=W8A8, Zero-shot=true2023.05 | 74.6 | — | — | |
| Our Trained ModelModel Size=6B, Data type=W4, Zero-shot=true2023.05 | 74 | — | — | |
| FIXED_E=16Experts=162026.05 | 74 | — | — | |
| EMO (Stage 3)E=16→322026.05 | 74 | — | — | |
| Our Trained ModelModel Size=13B, Data type=W4, Zero-shot=true2023.05 | 73.6 | — | — | |
| EMO (Stage 2)E=8→162026.05 | 73 | — | — | |
| SYNPROModel Scale=1.1B, Organic Tokens=2.2B2026.05 | 73 | — | — | |
| ReProModel Scale=1.1B, Organic Tokens=2.2B2026.05 | 72 | — | — | |
| DCModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 71 | — | — | |
| SYNPROModel Scale=400M, Organic Tokens=0.8B2026.05 | 70 | — | — | |
| UniBiasModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 69 | — | — | |
| ReProModel Scale=400M, Organic Tokens=0.8B2026.05 | 69 | — | — | |
| EMO (Stage 1)E=82026.05 | 68 | — | — | |
| EMO (Stage 4)E=32→642026.05 | 68 | — | — | |
| ICRBackbone=Llama2-7B2025.09 | 68 | — | — | |
| CCModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 67.8 | — | — | |
| PCModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 67.8 | — | — | |
| ICLModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 67.6 | — | — | |
| ICLModel=Llama-2 13b, ICL Setting=1-shot2024.05 | 67.5 | — | — | |
| Our Trained ModelModel Size=410M, Data type=FP16, Zero-shot=true2023.05 | 65.4 | — | — | |
| LIVEBackbone=Llama2-7B2025.09 | 64.8 | — | — | |
| M²IVBackbone=Llama2-7B2025.09 | 64.8 | — | — | |
| Our Trained ModelModel Size=410M, Data type=W8A8, Zero-shot=true2023.05 | 64 | — | — | |
| I2CLBackbone=Llama2-7B2025.09 | 63.6 | — | — | |
| RepeatModel Scale=1.1B, Organic Tokens=2.2B2026.05 | 63 | — | — | |
| Zero-shotBackbone=Llama2-7B2025.09 | 63 | — | — | |
| Our Trained ModelModel Size=410M, Data type=W4, Zero-shot=true2023.05 | 62.8 | — | — | |
| CCSModel=microsoft/deberta-xxlarge-v2-mnli2023.10 | 61 | — | — | |
| DCModel=Llama-2 7b, ICL Setting=1-shot2024.05 | 60.4 | — | — | |
| RepeatModel Scale=400M, Organic Tokens=0.8B2026.05 | 60 | — | — | |
| OPT 1.3BScale=1.3B2022.12 | — | 77 | 75 | |
| OPT 175BScale=175B2022.12 | — | 87 | 91.4 | |
| OPT 30BScale=30B2022.12 | — | 84 | 88.6 | |
| OPT-IML 1.3BScale=1.3B2022.12 | — | 77 | 78 | |
| OPT-IML 175BScale=175B2022.12 | — | 88 | 89 | |
| OPT-IML 30BScale=30B2022.12 | — | 85 | 89 |