Machine Reading Comprehension on MultiRC (dev)
77.5F1 ScoreGLaM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GLaMModel Size=64B/64E, Evaluation Protocol=Few-shot, Shots=42021.12 | 77.5 | — | — | — | |
| GPT-3Model Size=175B, Evaluation Protocol=Few-shot, Shots=322021.12 | 74.8 | — | — | — | |
| GLaMModel Size=64B/64E, Evaluation Protocol=One-shot2021.12 | 74.7 | — | — | — | |
| GLaMModel Size=64B/64E, Evaluation Protocol=Zero-shot2021.12 | 73.7 | — | — | — | |
| GPT-3Model Size=175B, Evaluation Protocol=Zero-shot2021.12 | 72.9 | — | — | — | |
| GPT-3Model Size=175B, Evaluation Protocol=One-shot2021.12 | 72.9 | — | — | — | |
| KT-NET BOTHKnowledge Source=WordNet + NELL, Implementation=Authors' own implementation2021.09 | 71.7 | — | — | 26.7 | |
| KELM BothKnowledge Source=WordNet + NELL2021.09 | 71 | — | — | 30.3 | |
| KELM WordNetKnowledge Source=WordNet2021.09 | 70.6 | — | — | 29.2 | |
| KELM NELLKnowledge Source=NELL2021.09 | 70.4 | — | — | 27.3 | |
| GPTPre-trained Source Task=RACE2018.10 | — | 69.3 | 67.2 | 15.2 | |
| GPT (2x)Pre-trained Source Task=RACE, Scale=2x2018.10 | — | 70.3 | 67.7 | 16.5 | |
| GPT+StrategiesPre-trained Source Task=RACE, Strategies=Proposed Strategies2018.10 | — | 71.5 | 69.2 | 22.6 | |
| GPT+Strategies (2x)Pre-trained Source Task=RACE, Strategies=Proposed Strategies, Scale=2x2018.10 | — | 73.1 | 70.5 | 21.8 | |
| Khashabi et al. (2018)2018.10 | — | 66.5 | 63.2 | 11.8 |