List your company
ResearchFormerly WizWand

Reading Comprehension on QuAC

74.4F1 ScoreFine-tuned SOTA
16.57631.58846.661.612May 28, 2020Dec 5, 2020Jun 14, 2021Dec 22, 2021Jul 1, 2022Jan 8, 2023Jul 18, 2023
Updated 8mo ago

Evaluation results

MethodLinks
Fine-tuned SOTASetting=Fine-tuned2020.0574.4
Llama 2Size=70B, shot=1-shot2023.0749.3
Llama 2Size=13B, shot=1-shot2023.0744.8
Llama 2Size=34B, shot=1-shot2023.0744.4
GPT-3Setting=Few-Shot2020.0544.3
Llama 1Size=33B, shot=0-shot2023.0744.1
GPT-3Setting=One-Shot2020.0543.3
FalconSize=40B, shot=1-shot2023.0743.3
Llama 2Size=34B, shot=0-shot2023.0742.9
Llama 2Size=13B, shot=0-shot2023.0742.7
Llama 2Size=70B, shot=0-shot2023.0742.4
GPT-3Setting=Zero-Shot2020.0541.5
FalconSize=40B, shot=0-shot2023.0741.2
MPTSize=30B, shot=1-shot2023.0741.1
Llama 1Size=65B, shot=0-shot2023.0741
MPTSize=30B, shot=0-shot2023.0740.4
Llama 1Size=33B, shot=1-shot2023.0740.3
Llama 1Size=13B, shot=0-shot2023.0739.9
Llama 1Size=65B, shot=1-shot2023.0739.8
Llama 2Size=7B, shot=1-shot2023.0739.7
Llama 2Size=7B, shot=0-shot2023.0739.4
Llama 1Size=7B, shot=0-shot2023.0738.9
MPTSize=7B, shot=0-shot2023.0738
MPTSize=7B, shot=1-shot2023.0737.7
Llama 1Size=13B, shot=1-shot2023.0736.5
Llama 1Size=7B, shot=1-shot2023.0732
FalconSize=7B, shot=0-shot2023.0724
FalconSize=7B, shot=1-shot2023.0718.8