Symbolic Reasoning on Coin Flip (Accuracy, Average Accuracy)
80.75AccuracyRM-Primed
Evaluation results
| Method | Links | ||
|---|---|---|---|
| RM-PrimedModel=Llama3-8B2026.03 | 80.75 | 68.23 | |
| RM-PrimedModel=GPT-3.52026.03 | 79.75 | 66.01 | |
| RM-Primed (R+)Model=Llama3-8B, R+ selection=only correct entries used2026.03 | 78.75 | 65.94 | |
| RM-Primed (R+)Model=GPT-3.5, R+ selection=only correct entries used2026.03 | 78 | 64.99 | |
| Few-shot CoTModel=GPT-3.52026.03 | 74.25 | 61.15 | |
| Contrastive CoTModel=GPT-3.5, Reflection=with2026.03 | 69.25 | 60.33 | |
| Contrastive CoTModel=Llama3-8B, Reflection=with2026.03 | 66.25 | 61.13 | |
| Few-shot CoTModel=Llama3-8B2026.03 | 65.75 | 64.19 | |
| Contrastive CoTModel=Llama3-8B, Reflection=without2026.03 | 60.75 | 59.56 | |
| Contrastive CoTModel=GPT-3.5, Reflection=without2026.03 | 59 | 57.01 |