Causal Inference on CounterBench
91.8AccuracyCoIn
Evaluation results
| Method | Links | |
|---|---|---|
| CoInModel=Deepseek-V32025.02 | 91.8 | |
| CoInModel=Gemini-1.5-flash2025.02 | 89.9 | |
| CoInModel=Claude-3 (Sonnet)2025.02 | 89.8 | |
| CoInModel=GPT-4o2025.02 | 89.4 | |
| CoInModel=Gemini-1.5-flash-8b2025.02 | 83.9 | |
| CoInModel=GPT-4o mini2025.02 | 79.9 | |
| CoInModel=Claude-3.5 (Haiku)2025.02 | 79.1 | |
| CDCRBase Model=Olmo3-7B-Instruct, Data=CDCR2026.05 | 77.1 | |
| CausalCoTModel=GPT-4o2025.02 | 75.8 | |
| CausalCoTModel=Deepseek-V32025.02 | 74.2 | |
| UNICOBase Model=Qwen3-4B, Data=UNICO2026.05 | 73.2 | |
| CausalCoTModel=Gemini-1.5-flash2025.02 | 71 | |
| CDCRBase Model=Qwen3-4B, Data=CDCR2026.05 | 70.4 | |
| UNICOBase Model=Qwen3-8B, Data=UNICO2026.05 | 70.2 | |
| Qwen3-32BBase Model=Qwen3-32B2026.05 | 69.9 | |
| Olmo3.1-32B-InstructBase Model=Olmo3.1-32B-Instruct2026.05 | 68.3 | |
| StandardModel=Gemini-1.5-flash2025.02 | 68 | |
| OriginalBase Model=Qwen3-4B, Data=Original2026.05 | 67.6 | |
| OriginalBase Model=Qwen3-8B, Data=Original2026.05 | 66.1 | |
| CDCRBase Model=Qwen3-8B, Data=CDCR2026.05 | 65.2 | |
| CausalCoTModel=Gemini-1.5-flash-8b2025.02 | 64.8 | |
| CauGymBase Model=Qwen3-8B, Data=CauGym2026.05 | 64.6 | |
| CausalCoTModel=Claude-3.5 (Haiku)2025.02 | 64.2 | |
| CauGymBase Model=Qwen3-4B, Data=CauGym2026.05 | 62.8 | |
| CauGymBase Model=Olmo3-7B-Instruct, Data=CauGym2026.05 | 60.2 | |
| CausalCoTModel=GPT-4o mini2025.02 | 59.8 | |
| UNICOBase Model=Olmo3-7B-Instruct, Data=UNICO2026.05 | 59.5 | |
| CausalCoTModel=Claude-3 (Sonnet)2025.02 | 59 | |
| CoInModel=GPT-3.5 turbo2025.02 | 58.9 | |
| OriginalBase Model=Olmo3-7B-Instruct, Data=Original2026.05 | 58.2 | |
| StandardModel=GPT-4o2025.02 | 52.8 | |
| SolverModel=Gemini-1.5-flash2025.02 | 52.8 | |
| StandardModel=GPT-3 (Davinci-002)2025.02 | 51.9 | |
| StandardModel=Deepseek-V32025.02 | 51.9 | |
| SolverModel=Claude-3 (Sonnet)2025.02 | 51.8 | |
| StandardModel=Claude-3 (Sonnet)2025.02 | 51.6 | |
| SolverModel=GPT-4o2025.02 | 51.4 | |
| StandardModel=GPT-3.5 turbo2025.02 | 50.6 | |
| StandardModel=GPT-4o mini2025.02 | 50.4 | |
| StandardModel=Gemini-1.5-flash-8b2025.02 | 50.4 | |
| SolverModel=Gemini-1.5-flash-8b2025.02 | 50.3 | |
| SolverModel=GPT-3 (Davinci-002)2025.02 | 50.1 | |
| CoInModel=GPT-3 (Davinci-002)2025.02 | 49.6 | |
| StandardModel=GPT-3 (Babbage-002)2025.02 | 49.6 | |
| SolverModel=GPT-3.5 turbo2025.02 | 49.6 | |
| CausalCoTModel=GPT-3 (Davinci-002)2025.02 | 49.3 | |
| SolverModel=Deepseek-V32025.02 | 49.3 | |
| CausalCoTModel=GPT-3.5 turbo2025.02 | 49.1 | |
| SolverModel=Claude-3.5 (Haiku)2025.02 | 48.3 | |
| SolverModel=GPT-3 (Babbage-002)2025.02 | 47.9 | |
| SolverModel=GPT-4o mini2025.02 | 47.2 | |
| CoInModel=GPT-3 (Babbage-002)2025.02 | 45.8 | |
| StandardModel=Claude-3.5 (Haiku)2025.02 | 39.8 | |
| CausalCoTModel=GPT-3 (Babbage-002)2025.02 | 9.8 |