Biomedical Knowledge Probing on BioLAMA
4.82CTD Accuracy @1OptiPrompt
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| OptiPromptModel=BioBERT2021.10 | 4.82 | 9.74 | 4.21 | 12.91 | 5.08 | 13.28 | — | |
| OptiPromptModel=BERT2021.10 | 3.56 | 6.97 | 3.29 | 8.13 | 1.44 | 3.65 | — | |
| OptiPromptModel=Bio-LM2021.10 | 2.99 | 10.19 | 10.6 | 25.15 | 8.25 | 20.19 | — | |
| Contrastive-ProbeModel=PubMedBERT2021.10 | 2.6 | 8.87 | 10.2 | 35.14 | 4.93 | 18.33 | — | |
| Contrastive-ProbeModel=Bio-LM2021.10 | 1.7 | 4.26 | 4.32 | 18.74 | 1.27 | 5.01 | — | |
| Contrastive-ProbeModel=BlueBERT2021.10 | 1.62 | 5.84 | 6.64 | 25.97 | 2.63 | 11.46 | — | |
| Mask PredictModel=Bio-LM2021.10 | 1.17 | 7.3 | 11.97 | 25.92 | 3.44 | 8.88 | — | |
| Mask PredictModel=BioBERT2021.10 | 0.42 | 3.25 | 3.67 | 11.2 | 1.16 | 3.82 | — | |
| Contrastive-ProbeModel=BioBERT2021.10 | 0.2 | 0.99 | 1.04 | 4.51 | 0.89 | 3.89 | — | |
| Mask PredictModel=BERT2021.10 | 0.06 | 1.2 | 1.16 | 6.04 | 0.82 | 1.99 | — | |
| BLOOMParams (bn)=176, zero-shot=true2022.11 | — | — | — | — | — | — | 9.7 | |
| GAL 1.3BParams (bn)=1.3, zero-shot=true2022.11 | — | — | — | — | — | — | 7.2 | |
| GAL 120BParams (bn)=120, zero-shot=true2022.11 | — | — | — | — | — | — | 8 | |
| GAL 125MParams (bn)=0.1, zero-shot=true2022.11 | — | — | — | — | — | — | 3.1 | |
| GAL 30BParams (bn)=30, zero-shot=true2022.11 | — | — | — | — | — | — | 6.9 | |
| GAL 6.7BParams (bn)=6.7, zero-shot=true2022.11 | — | — | — | — | — | — | 7.9 | |
| GPT-3 (text-davinci-002)Params (bn)=?, zero-shot=true2022.11 | — | — | — | — | — | — | 8.4 | |
| OPTParams (bn)=175, zero-shot=true2022.11 | — | — | — | — | — | — | 7.1 |