Causal Reasoning on CLadder
99.89Exact MatchARYA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| ARYAPrompting Strategy=Zero-shot prompting, Parameters=02026.03 | 99.89 | — | — | |
| Llama3.1-8B-InstructParameters=8B, Backbone=Llama 3.1, Variant=Instruct2026.01 | 88 | 66 | 90 | |
| Claude Opus 4.6Prompting Strategy=Optimized prompting / few-shot / CoT2026.03 | 87.2 | — | — | |
| Gemma-7B-itParameters=7B, Backbone=Gemma, Variant=Instruct2026.01 | 80 | 58 | 84 | |
| GPT-4Context=Best Published Baseline2026.03 | 76.4 | — | — | |
| GPT-5.2Prompting Strategy=Zero-shot prompting2026.03 | 67.8 | — | — | |
| Mistral-7BParameters=7B, Backbone=Mistral2026.01 | 58 | 80 | 94 | |
| Llama3.1-8BParameters=8B, Backbone=Llama 3.12026.01 | 57 | 60 | 73 | |
| Claude Opus 4.6Prompting Strategy=Zero-shot prompting2026.03 | 50.9 | — | — |