Logical Reasoning on FOLIO-wiki-curated (test)
98.04AccuracyCR
Evaluation Results
| Method | Links | |
|---|---|---|
| CRModel=GPT-4, n=22023.08 | 98.04 | |
| CoT-SCModel=GPT-4, k=162023.08 | 96.09 | |
| CoTModel=GPT-42023.08 | 95 | |
| DirectModel=GPT-42023.08 | 89.57 | |
| CRModel=LLaMA-65B, n=22023.08 | 79.57 | |
| CoT-SCModel=LLaMA-65B, k=162023.08 | 79.13 | |
| CRModel=GPT-3.5-turbo, n=22023.08 | 78.7 | |
| DirectModel=LLaMA-65B2023.08 | 74.78 | |
| CoTModel=LLaMA-65B2023.08 | 74.13 | |
| CoTModel=GPT-3.5-turbo2023.08 | 70.65 | |
| DirectModel=GPT-3.5-turbo2023.08 | 69.57 | |
| CoT-SCModel=GPT-3.5-turbo, k=162023.08 | 69.32 | |
| CRModel=LLaMA 13B, n=22023.08 | 55.87 | |
| CoT-SCModel=LLaMA 13B, k=162023.08 | 53.7 | |
| CoTModel=LLaMA 13B2023.08 | 52.17 | |
| DirectModel=LLaMA 13B2023.08 | 49.13 | |
| [Random]Model=Baseline2023.08 | 33.33 |