Causal Question Answering on CausalProbe-E
76.2EMG²-Reasoner
Evaluation results
| Method | Links | |
|---|---|---|
| G²-ReasonerBackbone=Claude 3 opus, Temperature=1.02025.06 | 76.2 | |
| VanillaModel=Claude 32025.06 | 75.8 | |
| G²-ReasonerModel=Claude 32025.06 | 75.8 | |
| RAGBackbone=Claude 3 opus, Temperature=1.02025.06 | 75.6 | |
| RAGModel=Claude 32025.06 | 75.6 | |
| VanillaBackbone=Claude 3 opus, Temperature=1.02025.06 | 75.5 | |
| CoTModel=Claude 32025.06 | 75.3 | |
| CoTBackbone=Claude 3 opus, Temperature=1.02025.06 | 75.1 | |
| G²-ReasonerBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 74.7 | |
| G²-ReasonerModel=GPT 3.52025.06 | 74.6 | |
| RAGBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 74.3 | |
| RAGModel=GPT 3.52025.06 | 74.1 | |
| CoTBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 74 | |
| CoTModel=GPT 3.52025.06 | 73.7 | |
| VanillaModel=GPT 3.52025.06 | 73.2 | |
| VanillaBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 73 | |
| CoTBackbone=LLaMA 32025.06 | 72 | |
| CoTModel=LLaMA 3 8B2025.06 | 72 | |
| G²-ReasonerBackbone=LLaMA 32025.06 | 71.8 | |
| G²-ReasonerModel=LLaMA 3 8B2025.06 | 71.8 | |
| VanillaBackbone=LLaMA 32025.06 | 71.5 | |
| VanillaModel=LLaMA 3 8B2025.06 | 71.5 | |
| RAGBackbone=LLaMA 32025.06 | 70.4 | |
| RAGModel=LLaMA 3 8B2025.06 | 70.4 | |
| G²-ReasonerBackbone=LLaMA 22025.06 | 64.2 | |
| G²-ReasonerModel=LLaMA 2 7B2025.06 | 64.2 | |
| CoTBackbone=LLaMA 22025.06 | 63.6 | |
| CoTModel=LLaMA 2 7B2025.06 | 63.6 | |
| RAGBackbone=LLaMA 22025.06 | 62.1 | |
| RAGModel=LLaMA 2 7B2025.06 | 62.1 | |
| VanillaBackbone=LLaMA 22025.06 | 61.6 | |
| VanillaModel=LLaMA 2 7B2025.06 | 61.6 |