Question Answering on QA4MRE 2013
42.6AccuracyGPT-3
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-3Model Variant=DaVinci, Zero-shot=true2022.04 | 42.6 | |
| FairSeqNumber of Parameters=6.7B, Evaluation Protocol=5-shot2022.04 | 39.1 | |
| FairSeqNumber of Parameters=13B, Evaluation Protocol=5-shot2022.04 | 38.7 | |
| FairSeqNumber of Parameters=2.7B, Zero-Shot=true2022.04 | 38 | |
| FairSeqNumber of Parameters=13B, Zero-Shot=true2022.04 | 38 | |
| GPT-3Model Variant=Curie, Zero-shot=true2022.04 | 37.7 | |
| GPT-JModel Size=6B, Zero-shot=true2022.04 | 37.3 | |
| FairSeqNumber of Parameters=6.7B, Zero-Shot=true2022.04 | 37 | |
| GPT-3Model Variant=Babbage, Zero-shot=true2022.04 | 37 | |
| GPT-JParameters=6B, Evaluation=Five-shot2022.04 | 36.6 | |
| GPT-NeoXParameters=20B, Evaluation=Five-shot2022.04 | 36.3 | |
| GPT-NeoXModel Size=20B, Zero-shot=true2022.04 | 36.3 | |
| FairSeqNumber of Parameters=2.7B, Evaluation Protocol=5-shot2022.04 | 35.2 | |
| FairSeqNumber of Parameters=355M, Evaluation Protocol=5-shot2022.04 | 33.8 | |
| FairSeqNumber of Parameters=1.3B, Evaluation Protocol=5-shot2022.04 | 33.8 | |
| FairSeqNumber of Parameters=355M, Zero-Shot=true2022.04 | 33.5 | |
| FairSeqNumber of Parameters=1.3B, Zero-Shot=true2022.04 | 32.7 | |
| FairSeqNumber of Parameters=125M, Evaluation Protocol=5-shot2022.04 | 32.4 | |
| GPT-3Model Variant=Ada, Zero-shot=true2022.04 | 32 | |
| FairSeqNumber of Parameters=125M, Zero-Shot=true2022.04 | 28.5 |