List your company
ResearchFormerly WizWand

Reasoning and Question Answering on BoolQ, RTE, HellaSWAG, ARC, OpenBookQA, and PiQA

67.24Avg AccuracyBefore finetune
34.32442.869551.41559.9605Jun 12, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Before finetuneBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0667.24
Target LLMBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0666.93
ULDBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0666.85
NPO+GDBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0661.77
NPO+KLBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0661.14
Offset-NPO+KLBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0658.72
GA+GDBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0658.34
Offset-DPO+KLBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0656.59
DPO+KLBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0656.34
GA+KLBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0655.41
NPOBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0654.73
DPO+GDBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0653.91
Offset-GA+KLBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0653.78
DPOBackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0648.12
GABackbone=Mistral-7B-instruct, Mode=Zero-shot2024.0635.59