List your company
ResearchFormerly WizWand

Boolean Question Answering on BoolQ (Calibrated Accuracy)

86.1Calibrated AccuracySC+IC (tune)
60.41267.08173.7580.419May 29, 2024
Updated 8mo ago

Evaluation results

MethodLinks
SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=L2M2024.0586.1
SC+IC (tune)Backbone=MIXTRAL-8×7B, Prompting Strategy=CoT2024.0581.8
SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=CoT2024.0581.4
SC+IC (tune)Backbone=LLAMA-2-13B, Prompting Strategy=L2M2024.0580.4
SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=L2M2024.0579.5
SC+IC (tune)Backbone=MISTRAL-7B, Prompting Strategy=CoT2024.0577.8
SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=L2M2024.0573.5
SC+IC (transfer)Backbone=LLAMA-2-7B, Prompting Strategy=L2M2024.0573.5
SC+ICBackbone=LLAMA-2-7B, Prompting Strategy=L2M2024.0573.4
SCBackbone=LLAMA-2-7B, Prompting Strategy=L2M, Sampled reasoning paths=402024.0573.2
SC+ABackbone=LLAMA-2-7B, Prompting Strategy=L2M2024.0573
SC+IC (tune)Backbone=LLAMA-2-7B, Prompting Strategy=CoT2024.0571.5
SC+IC (transfer)Backbone=LLAMA-2-7B, Prompting Strategy=CoT2024.0571.5
SC+ICBackbone=LLAMA-2-7B, Prompting Strategy=CoT2024.0570.7
SCBackbone=LLAMA-2-7B, Prompting Strategy=CoT, Sampled reasoning paths=402024.0570
SC+ABackbone=LLAMA-2-7B, Prompting Strategy=CoT2024.0569.9
GreedyBackbone=LLAMA-2-7B, Prompting Strategy=CoT2024.0567.1
GreedyBackbone=LLAMA-2-7B, Prompting Strategy=L2M2024.0561.4