Deductive Reasoning on ProofWriter
92.1Calibrated AccuracySC+IC (tune)
Evaluation results
| Method | Links | |
|---|---|---|
| SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=L2M2024.05 | 92.1 | |
| SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=L2M2024.05 | 89.6 | |
| SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=L2M2024.05 | 88.6 | |
| SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=CoT2024.05 | 78.8 | |
| SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=CoT2024.05 | 71.2 | |
| SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=L2M2024.05 | 70.7 | |
| SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=CoT2024.05 | 62.9 | |
| SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=CoT2024.05 | 54.1 |