Causal Question Answering on CausalProbe-H
0.701EMG²-Reasoner
Evaluation Results
| Method | Links | |
|---|---|---|
| G²-ReasonerBackbone=Claude 3 opus, Temperature=1.02025.06 | 0.701 | |
| CoTModel=Claude 32025.06 | 0.701 | |
| CoTBackbone=Claude 3 opus, Temperature=1.02025.06 | 0.7 | |
| G²-ReasonerModel=Claude 32025.06 | 0.696 | |
| G²-ReasonerModel=GPT 3.52025.06 | 0.693 | |
| VanillaModel=Claude 32025.06 | 0.692 | |
| G²-ReasonerBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 0.69 | |
| VanillaBackbone=Claude 3 opus, Temperature=1.02025.06 | 0.688 | |
| RAGBackbone=Claude 3 opus, Temperature=1.02025.06 | 0.687 | |
| RAGModel=Claude 32025.06 | 0.685 | |
| RAGBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 0.678 | |
| RAGModel=GPT 3.52025.06 | 0.678 | |
| VanillaModel=GPT 3.52025.06 | 0.671 | |
| VanillaBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 0.67 | |
| CoTModel=GPT 3.52025.06 | 0.667 | |
| CoTBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 0.662 | |
| G²-ReasonerBackbone=LLaMA 32025.06 | 0.658 | |
| G²-ReasonerModel=LLaMA 3 8B2025.06 | 0.658 | |
| RAGBackbone=LLaMA 32025.06 | 0.655 | |
| RAGModel=LLaMA 3 8B2025.06 | 0.655 | |
| VanillaBackbone=LLaMA 32025.06 | 0.652 | |
| VanillaModel=LLaMA 3 8B2025.06 | 0.652 | |
| CoTBackbone=LLaMA 32025.06 | 0.644 | |
| CoTModel=LLaMA 3 8B2025.06 | 0.644 | |
| G²-ReasonerBackbone=LLaMA 22025.06 | 0.582 | |
| G²-ReasonerModel=LLaMA 2 7B2025.06 | 0.582 | |
| RAGBackbone=LLaMA 22025.06 | 0.575 | |
| RAGModel=LLaMA 2 7B2025.06 | 0.575 | |
| CoTBackbone=LLaMA 22025.06 | 0.573 | |
| CoTModel=LLaMA 2 7B2025.06 | 0.573 | |
| VanillaBackbone=LLaMA 22025.06 | 0.565 | |
| VanillaModel=LLaMA 2 7B2025.06 | 0.565 |