Causal Question Answering on COPA
99.3EMG²-Reasoner
Evaluation results
| Method | Links | |
|---|---|---|
| G²-ReasonerBackbone=Claude 3 opus, Temperature=1.02025.06 | 99.3 | |
| VanillaBackbone=Claude 3 opus, Temperature=1.02025.06 | 99.2 | |
| CoTBackbone=Claude 3 opus, Temperature=1.02025.06 | 99.1 | |
| VanillaModel=Claude 32025.06 | 99.1 | |
| CoTModel=Claude 32025.06 | 99.1 | |
| RAGModel=Claude 32025.06 | 99 | |
| G²-ReasonerModel=Claude 32025.06 | 99 | |
| RAGBackbone=Claude 3 opus, Temperature=1.02025.06 | 98.8 | |
| G²-ReasonerBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 95.3 | |
| G²-ReasonerModel=GPT 3.52025.06 | 95.3 | |
| CoTBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 95.1 | |
| CoTModel=GPT 3.52025.06 | 95.1 | |
| G²-ReasonerBackbone=LLaMA 32025.06 | 94.8 | |
| G²-ReasonerModel=LLaMA 3 8B2025.06 | 94.8 | |
| VanillaModel=GPT 3.52025.06 | 94.8 | |
| CoTBackbone=LLaMA 32025.06 | 94.4 | |
| CoTModel=LLaMA 3 8B2025.06 | 94.4 | |
| VanillaBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 94.3 | |
| RAGBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 93.9 | |
| VanillaBackbone=LLaMA 32025.06 | 93.7 | |
| VanillaModel=LLaMA 3 8B2025.06 | 93.7 | |
| RAGModel=GPT 3.52025.06 | 93.6 | |
| RAGBackbone=LLaMA 32025.06 | 91.2 | |
| RAGModel=LLaMA 3 8B2025.06 | 91.2 | |
| G²-ReasonerBackbone=LLaMA 22025.06 | 81.3 | |
| G²-ReasonerModel=LLaMA 2 7B2025.06 | 81.3 | |
| CoTBackbone=LLaMA 22025.06 | 81.2 | |
| CoTModel=LLaMA 2 7B2025.06 | 81.2 | |
| RAGBackbone=LLaMA 22025.06 | 75.7 | |
| RAGModel=LLaMA 2 7B2025.06 | 75.7 | |
| VanillaBackbone=LLaMA 22025.06 | 75.2 | |
| VanillaModel=LLaMA 2 7B2025.06 | 75.2 |