Boolean Question Answering on BoolQ (Calibrated Accuracy)
86.1Calibrated AccuracySC+IC (tune)
Evaluation Results
| Method | Links | |
|---|---|---|
| SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=L2M2024.05 | 86.1 | |
| SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=CoT2024.05 | 81.8 | |
| SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=CoT2024.05 | 81.4 | |
| SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=L2M2024.05 | 80.4 | |
| SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=L2M2024.05 | 79.5 | |
| SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=CoT2024.05 | 77.8 | |
| SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=L2M2024.05 | 73.5 | |
| SC+IC (transfer)Backbone=LLAMA-2-7B, Prompting Strategy=L2M2024.05 | 73.5 | |
| SC+ICBackbone=LLAMA-2-7B, Prompting Strategy=L2M2024.05 | 73.4 | |
| SCBackbone=LLAMA-2-7B, Prompting Strategy=L2M, Sampled reasoning paths=402024.05 | 73.2 | |
| SC+ABackbone=LLAMA-2-7B, Prompting Strategy=L2M2024.05 | 73 | |
| SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=CoT2024.05 | 71.5 | |
| SC+IC (transfer)Backbone=LLAMA-2-7B, Prompting Strategy=CoT2024.05 | 71.5 | |
| SC+ICBackbone=LLAMA-2-7B, Prompting Strategy=CoT2024.05 | 70.7 | |
| SCBackbone=LLAMA-2-7B, Prompting Strategy=CoT, Sampled reasoning paths=402024.05 | 70 | |
| SC+ABackbone=LLAMA-2-7B, Prompting Strategy=CoT2024.05 | 69.9 | |
| GreedyBackbone=LLAMA-2-7B, Prompting Strategy=CoT2024.05 | 67.1 | |
| GreedyBackbone=LLAMA-2-7B, Prompting Strategy=L2M2024.05 | 61.4 |