Symbolic Reasoning on CoinFlip
100Calibrated AccuracySC+IC (tune)
Evaluation Results
| Method | Links | |
|---|---|---|
| SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=L2M2024.05 | 100 | |
| SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=L2M2024.05 | 99.9 | |
| SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=CoT2024.05 | 99.9 | |
| SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=CoT2024.05 | 97.7 | |
| SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=L2M2024.05 | 96.5 | |
| SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=L2M2024.05 | 93.9 | |
| SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=CoT2024.05 | 86.2 | |
| SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=CoT2024.05 | 78.6 |