Causal Question Answering on e-CARE
86.4Exact MatchCoT
Evaluation results
| Method | Links | |
|---|---|---|
| CoTModel=Claude 32025.06 | 86.4 | |
| CoTBackbone=Claude 3 opus, Temperature=1.02025.06 | 86.3 | |
| VanillaModel=Claude 32025.06 | 86.1 | |
| VanillaBackbone=Claude 3 opus, Temperature=1.02025.06 | 85.7 | |
| G²-ReasonerBackbone=Claude 3 opus, Temperature=1.02025.06 | 85.4 | |
| G²-ReasonerModel=Claude 32025.06 | 84.9 | |
| RAGBackbone=Claude 3 opus, Temperature=1.02025.06 | 84.2 | |
| RAGModel=Claude 32025.06 | 83.6 | |
| G²-ReasonerBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 82.1 | |
| G²-ReasonerModel=GPT 3.52025.06 | 82.1 | |
| VanillaModel=GPT 3.52025.06 | 81.4 | |
| VanillaBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 81.1 | |
| RAGModel=GPT 3.52025.06 | 80.9 | |
| CoTBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 80.7 | |
| RAGBackbone=GPT 3.5 turbo, Temperature=1.02025.06 | 80.6 | |
| CoTModel=GPT 3.52025.06 | 80.2 | |
| G²-ReasonerBackbone=LLaMA 32025.06 | 77.9 | |
| G²-ReasonerModel=LLaMA 3 8B2025.06 | 77.9 | |
| VanillaBackbone=LLaMA 32025.06 | 77.8 | |
| VanillaModel=LLaMA 3 8B2025.06 | 77.8 | |
| CoTBackbone=LLaMA 32025.06 | 77 | |
| CoTModel=LLaMA 3 8B2025.06 | 77 | |
| RAGBackbone=LLaMA 32025.06 | 76 | |
| RAGModel=LLaMA 3 8B2025.06 | 76 | |
| G²-ReasonerBackbone=LLaMA 22025.06 | 70.1 | |
| G²-ReasonerModel=LLaMA 2 7B2025.06 | 70.1 | |
| CoTBackbone=LLaMA 22025.06 | 69.7 | |
| CoTModel=LLaMA 2 7B2025.06 | 69.7 | |
| RAGBackbone=LLaMA 22025.06 | 68.7 | |
| RAGModel=LLaMA 2 7B2025.06 | 68.7 | |
| VanillaBackbone=LLaMA 22025.06 | 68.4 | |
| VanillaModel=LLaMA 2 7B2025.06 | 68.4 |