Logical Reasoning on LogiQA (Accuracy, Improvement)
50.94AccuracyRICP
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RICPBackbone=Qwen-Turbo, Prompting Strategy=Auto CoT2024.07 | 50.94 | 1.72 | |
| RICPBackbone=Qwen-Turbo, Prompting Strategy=Complex CoT2024.07 | 50.47 | 2.66 | |
| RICPBackbone=Qwen-Turbo, Prompting Strategy=Few-shot CoT2024.07 | 49.69 | 1.72 | |
| Auto CoTBackbone=Qwen-Turbo, Enhancement=None2024.07 | 49.22 | — | |
| RICPBackbone=Qwen-Turbo, Prompting Strategy=Standard Prompting2024.07 | 48.75 | 3.29 | |
| Few-shot CoTBackbone=Qwen-Turbo, Enhancement=None2024.07 | 47.96 | — | |
| Complex CoTBackbone=Qwen-Turbo, Enhancement=None2024.07 | 47.81 | — | |
| RICPBackbone=Qwen-Turbo, Prompting Strategy=Zero-shot CoT2024.07 | 47.34 | 5.02 | |
| RICPBackbone=GPT-3.5-Turbo, Prompting Strategy=Few-shot CoT2024.07 | 46.08 | 3.76 | |
| Standard PromptingBackbone=Qwen-Turbo, Enhancement=None2024.07 | 45.45 | — | |
| RICPBackbone=GPT-3.5-Turbo, Prompting Strategy=Auto CoT2024.07 | 44.67 | 1.1 | |
| RICPBackbone=GPT-3.5-Turbo, Prompting Strategy=Zero-shot CoT2024.07 | 43.73 | 1.1 | |
| Auto CoTBackbone=GPT-3.5-Turbo, Enhancement=None2024.07 | 43.57 | — | |
| RICPBackbone=GPT-3.5-Turbo, Prompting Strategy=Standard Prompting2024.07 | 43.42 | 3.13 | |
| Zero-shot CoTBackbone=GPT-3.5-Turbo, Enhancement=None2024.07 | 42.63 | — | |
| Zero-shot CoTBackbone=Qwen-Turbo, Enhancement=None2024.07 | 42.32 | — | |
| Few-shot CoTBackbone=GPT-3.5-Turbo, Enhancement=None2024.07 | 42.32 | — | |
| Standard PromptingBackbone=GPT-3.5-Turbo, Enhancement=None2024.07 | 40.28 | — | |
| RICPBackbone=GPT-3.5-Turbo, Prompting Strategy=Complex CoT2024.07 | 38.24 | 1.41 | |
| Complex CoTBackbone=GPT-3.5-Turbo, Enhancement=None2024.07 | 36.83 | — |