Question Answering on HeadQA English
40.1AccuracyPythia-6.9B
Evaluation results
| Method | Links | |
|---|---|---|
| Pythia-6.9BEvaluation protocol=Zero-shot, Parameters=6.9B2024.02 | 40.1 | |
| LLaMA2-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 39.5 | |
| LLaMA-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 38.7 | |
| Falcon-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 38.6 | |
| GPT-NeoXParameters=20B, Evaluation=Five-shot2022.04 | 38.5 | |
| MPT-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 37.4 | |
| OLMo-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 37.3 | |
| RPJ-INCITE-7BEvaluation protocol=Zero-shot, Parameters=7B2024.02 | 36.9 | |
| GPT-3Model Variant=DaVinci, Zero-shot=true2022.04 | 35.6 | |
| GPT-JParameters=6B, Evaluation=Five-shot2022.04 | 32.6 | |
| GPT-3Model Variant=Curie, Zero-shot=true2022.04 | 31.7 | |
| FairSeqNumber of Parameters=13B, Evaluation Protocol=5-shot2022.04 | 28.2 | |
| FairSeqNumber of Parameters=6.7B, Zero-Shot=true2022.04 | 28 | |
| FairSeqNumber of Parameters=13B, Zero-Shot=true2022.04 | 28 | |
| GPT-3Model Variant=Babbage, Zero-shot=true2022.04 | 27.8 | |
| FairSeqNumber of Parameters=6.7B, Evaluation Protocol=5-shot2022.04 | 27.6 | |
| FairSeqNumber of Parameters=2.7B, Evaluation Protocol=5-shot2022.04 | 26.6 | |
| FairSeqNumber of Parameters=2.7B, Zero-Shot=true2022.04 | 26.4 | |
| FairSeqNumber of Parameters=1.3B, Zero-Shot=true2022.04 | 25.6 | |
| FairSeqNumber of Parameters=1.3B, Evaluation Protocol=5-shot2022.04 | 25.4 | |
| GPT-3Model Variant=Ada, Zero-shot=true2022.04 | 24.5 | |
| FairSeqNumber of Parameters=355M, Evaluation Protocol=5-shot2022.04 | 24 | |
| FairSeqNumber of Parameters=125M, Evaluation Protocol=5-shot2022.04 | 23.5 | |
| FairSeqNumber of Parameters=125M, Zero-Shot=true2022.04 | 23.3 | |
| FairSeqNumber of Parameters=355M, Zero-Shot=true2022.04 | 23.3 |