Knowledge Probing on LAMA
13.9Google-RE P@1BERT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| BERT2022.02 | 13.9 | 34.9 | — | — | |
| BERT-PELTBackbone=BERT2022.02 | 13.3 | 40.7 | — | — | |
| ToolformerAPI Calls=Enabled2023.02 | 11.5 | 53.5 | — | 33.8 | |
| ERNIE2022.02 | 10 | 24.9 | — | — | |
| BERT2019.11 | 9.8 | 31.1 | 15.6 | 14.1 | |
| KEPLER-Wiki2019.11 | 7.3 | 24.6 | 18.7 | 14.3 | |
| KEPLER-W+W2019.11 | 7.3 | 24.4 | 17.6 | 10.8 | |
| Our RoBERTa2019.11 | 7 | 23.2 | 19 | 8 | |
| GPT-3Parameters=175B2023.02 | 7 | 39.8 | — | 26.8 | |
| BART-PELTBackbone=BART2022.02 | 6.9 | 24.4 | — | — | |
| PELTBackbone=RoBERTa2022.02 | 6.4 | 27.5 | — | — | |
| ToolformerAPI Calls=Disabled2023.02 | 6.3 | 34.9 | — | 22.1 | |
| GPT-J + CCTraining Data=Common Crawl2023.02 | 5.6 | 33.2 | — | 19.2 | |
| KEPLER2022.02 | 5.5 | 23.4 | — | — | |
| RoBERTa2022.02 | 5.4 | 24.7 | — | — | |
| RoBERTa2019.11 | 5.3 | 24.7 | 19.5 | 9.1 | |
| BART2022.02 | 5.1 | 21.2 | — | — | |
| GPT-JTraining Data=Standard2023.02 | 4.9 | 31.9 | — | 17.8 | |
| LUKE2022.02 | 3.8 | 32 | — | — | |
| OPTParameters=66B2023.02 | 2.9 | 30.1 | — | 21.6 |