Causal Question Answering on CausalNet
93.3EMVanilla
Evaluation results
| Method | Links | |
|---|---|---|
| VanillaModel=Claude 32025.06 | 93.3 | |
| VanillaBackbone=Claude 3 opus, Temperature=1.02025.06 | 93 | |
| G²-ReasonerBackbone=Claude 3 opus, Temperature=1.02025.06 | 92.9 | |
| G²-ReasonerModel=Claude 32025.06 | 92.9 | |
| RAGModel=Claude 32025.06 | 92.2 | |
| RAGBackbone=Claude 3 opus, Temperature=1.02025.06 | 91 | |
| CoTModel=Claude 32025.06 | 91 | |
| CoTBackbone=Claude 3 opus, Temperature=1.02025.06 | 90.7 | |
| G²-ReasonerBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 90.4 | |
| RAGModel=GPT 3.52025.06 | 89.8 | |
| G²-ReasonerModel=GPT 3.52025.06 | 89.8 | |
| VanillaModel=GPT 3.52025.06 | 89.7 | |
| VanillaBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 89.5 | |
| RAGBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 89.4 | |
| CoTModel=GPT 3.52025.06 | 87.4 | |
| CoTBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 87.2 | |
| RAGBackbone=LLaMA 32025.06 | 86 | |
| RAGModel=LLaMA 3 8B2025.06 | 86 | |
| VanillaBackbone=LLaMA 32025.06 | 85.7 | |
| VanillaModel=LLaMA 3 8B2025.06 | 85.7 | |
| G²-ReasonerBackbone=LLaMA 32025.06 | 85.5 | |
| G²-ReasonerModel=LLaMA 3 8B2025.06 | 85.5 | |
| CoTBackbone=LLaMA 32025.06 | 76.7 | |
| CoTModel=LLaMA 3 8B2025.06 | 76.7 | |
| G²-ReasonerBackbone=LLaMA 22025.06 | 68.1 | |
| G²-ReasonerModel=LLaMA 2 7B2025.06 | 68.1 | |
| VanillaBackbone=LLaMA 22025.06 | 67.3 | |
| VanillaModel=LLaMA 2 7B2025.06 | 67.3 | |
| CoTBackbone=LLaMA 22025.06 | 66.6 | |
| CoTModel=LLaMA 2 7B2025.06 | 66.6 | |
| RAGBackbone=LLaMA 22025.06 | 65 | |
| RAGModel=LLaMA 2 7B2025.06 | 65 |