# Causal Question Answering on COPA Source: [Causal Question Answering on COPA]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:41:31.942Z Evaluation of model performance on the COPA dataset for Causal Question Answering tasks, typically measured by Exact Match (EM). A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Causal Question Answering | | Dataset (including any stated split) | COPA | | Metric | EM | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 32 of 32 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [EM](): Higher is better; unit: PERCENT. ## Recorded results | Method | EM | Result date | Report paper | | --- | --- | --- | --- | | G²-Reasoner | 99.3 | Not recorded | 68fa79b31e209131cea7ef6d | | Vanilla | 99.2 | Not recorded | 68fa79b31e209131cea7ef6d | | CoT | 99.1 | Not recorded | 68fa79b31e209131cea7ef6d | | Vanilla | 99.1 | Not recorded | 68fa79b31e209131cea7ef6d | | CoT | 99.1 | Not recorded | 68fa79b31e209131cea7ef6d | | RAG | 99 | Not recorded | 68fa79b31e209131cea7ef6d | | G²-Reasoner | 99 | Not recorded | 68fa79b31e209131cea7ef6d | | RAG | 98.8 | Not recorded | 68fa79b31e209131cea7ef6d | | G²-Reasoner | 95.3 | Not recorded | 68fa79b31e209131cea7ef6d | | G²-Reasoner | 95.3 | Not recorded | 68fa79b31e209131cea7ef6d | | CoT | 95.1 | Not recorded | 68fa79b31e209131cea7ef6d | | CoT | 95.1 | Not recorded | 68fa79b31e209131cea7ef6d | | G²-Reasoner | 94.8 | Not recorded | 68fa79b31e209131cea7ef6d | | G²-Reasoner | 94.8 | Not recorded | 68fa79b31e209131cea7ef6d | | Vanilla | 94.8 | Not recorded | 68fa79b31e209131cea7ef6d | | CoT | 94.4 | Not recorded | 68fa79b31e209131cea7ef6d | | CoT | 94.4 | Not recorded | 68fa79b31e209131cea7ef6d | | Vanilla | 94.3 | Not recorded | 68fa79b31e209131cea7ef6d | | RAG | 93.9 | Not recorded | 68fa79b31e209131cea7ef6d | | Vanilla | 93.7 | Not recorded | 68fa79b31e209131cea7ef6d | | Vanilla | 93.7 | Not recorded | 68fa79b31e209131cea7ef6d | | RAG | 93.6 | Not recorded | 68fa79b31e209131cea7ef6d | | RAG | 91.2 | Not recorded | 68fa79b31e209131cea7ef6d | | RAG | 91.2 | Not recorded | 68fa79b31e209131cea7ef6d | | G²-Reasoner | 81.3 | Not recorded | 68fa79b31e209131cea7ef6d | | G²-Reasoner | 81.3 | Not recorded | 68fa79b31e209131cea7ef6d | | CoT | 81.2 | Not recorded | 68fa79b31e209131cea7ef6d | | CoT | 81.2 | Not recorded | 68fa79b31e209131cea7ef6d | | RAG | 75.7 | Not recorded | 68fa79b31e209131cea7ef6d | | RAG | 75.7 | Not recorded | 68fa79b31e209131cea7ef6d | | Vanilla | 75.2 | Not recorded | 68fa79b31e209131cea7ef6d | | Vanilla | 75.2 | Not recorded | 68fa79b31e209131cea7ef6d | ### G²-Reasoner EM: 99.3. Result ID: 698b59957a967eddb76ddbd2. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Claude 3 opus","Temperature":"1.0"}. This record does not assert independent reproduction. ### Vanilla EM: 99.2. Result ID: 698b59957a967eddb76ddbcf. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Claude 3 opus","Temperature":"1.0"}. This record does not assert independent reproduction. ### CoT EM: 99.1. Result ID: 698b59957a967eddb76ddbd0. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Claude 3 opus","Temperature":"1.0"}. This record does not assert independent reproduction. ### Vanilla EM: 99.1. Result ID: 698c535b37894ef7d9cd8c03. [Reporting paper]() Recorded evaluation configuration: {"Model":"Claude 3"}. This record does not assert independent reproduction. ### CoT EM: 99.1. Result ID: 698c535b37894ef7d9cd8c04. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Claude 3"}. This record does not assert independent reproduction. ### RAG EM: 99. Result ID: 698c535b37894ef7d9cd8c05. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Claude 3"}. This record does not assert independent reproduction. ### G²-Reasoner EM: 99. Result ID: 698c535b37894ef7d9cd8c06. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"Claude 3"}. This record does not assert independent reproduction. ### RAG EM: 98.8. Result ID: 698b59957a967eddb76ddbd1. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"Claude 3 opus","Temperature":"1.0"}. This record does not assert independent reproduction. ### G²-Reasoner EM: 95.3. Result ID: 698b59957a967eddb76ddbce. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"GPT 3.5 turbo","Temperature":"1.0"}. This record does not assert independent reproduction. ### G²-Reasoner EM: 95.3. Result ID: 698c535b37894ef7d9cd8c02. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT 3.5"}. This record does not assert independent reproduction. ### CoT EM: 95.1. Result ID: 698b59957a967eddb76ddbcc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"GPT 3.5 turbo","Temperature":"1.0"}. This record does not assert independent reproduction. ### CoT EM: 95.1. Result ID: 698c535b37894ef7d9cd8c00. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"GPT 3.5"}. This record does not assert independent reproduction. ### G²-Reasoner EM: 94.8. Result ID: 698b59957a967eddb76ddbca. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaMA 3"}. This record does not assert independent reproduction. ### G²-Reasoner EM: 94.8. Result ID: 698c535b37894ef7d9cd8bfe. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA 3 8B"}. This record does not assert independent reproduction. ### Vanilla EM: 94.8. Result ID: 698c535b37894ef7d9cd8bff. [Reporting paper]() Recorded evaluation configuration: {"Model":"GPT 3.5"}. This record does not assert independent reproduction. ### CoT EM: 94.4. Result ID: 698b59957a967eddb76ddbc8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3"}. This record does not assert independent reproduction. ### CoT EM: 94.4. Result ID: 698c535b37894ef7d9cd8bfc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA 3 8B"}. This record does not assert independent reproduction. ### Vanilla EM: 94.3. Result ID: 698b59957a967eddb76ddbcb. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"GPT 3.5 turbo","Temperature":"1.0"}. This record does not assert independent reproduction. ### RAG EM: 93.9. Result ID: 698b59957a967eddb76ddbcd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"GPT 3.5 turbo","Temperature":"1.0"}. This record does not assert independent reproduction. ### Vanilla EM: 93.7. Result ID: 698b59957a967eddb76ddbc7. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3"}. This record does not assert independent reproduction. ### Vanilla EM: 93.7. Result ID: 698c535b37894ef7d9cd8bfb. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA 3 8B"}. This record does not assert independent reproduction. ### RAG EM: 93.6. Result ID: 698c535b37894ef7d9cd8c01. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"GPT 3.5"}. This record does not assert independent reproduction. ### RAG EM: 91.2. Result ID: 698b59957a967eddb76ddbc9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaMA 3"}. This record does not assert independent reproduction. ### RAG EM: 91.2. Result ID: 698c535b37894ef7d9cd8bfd. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA 3 8B"}. This record does not assert independent reproduction. ### G²-Reasoner EM: 81.3. Result ID: 698b59957a967eddb76ddbc6. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaMA 2"}. This record does not assert independent reproduction. ### G²-Reasoner EM: 81.3. Result ID: 698c535b37894ef7d9cd8bfa. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA 2 7B"}. This record does not assert independent reproduction. ### CoT EM: 81.2. Result ID: 698b59957a967eddb76ddbc4. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 2"}. This record does not assert independent reproduction. ### CoT EM: 81.2. Result ID: 698c535b37894ef7d9cd8bf8. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"LLaMA 2 7B"}. This record does not assert independent reproduction. ### RAG EM: 75.7. Result ID: 698b59957a967eddb76ddbc5. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Backbone":"LLaMA 2"}. This record does not assert independent reproduction. ### RAG EM: 75.7. Result ID: 698c535b37894ef7d9cd8bf9. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Model":"LLaMA 2 7B"}. This record does not assert independent reproduction. ### Vanilla EM: 75.2. Result ID: 698b59957a967eddb76ddbc3. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 2"}. This record does not assert independent reproduction. ### Vanilla EM: 75.2. Result ID: 698c535b37894ef7d9cd8bf7. [Reporting paper]() Recorded evaluation configuration: {"Model":"LLaMA 2 7B"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.