Biomedical Knowledge Probing on MedLAMA Full Set 1.0
0.0571Acc@1Contrastive-Probe
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Contrastive-ProbePLM=PubMedBERT2021.10 | 0.0571 | 0.2431 | |
| Contrastive-ProbePLM=BlueBERT2021.10 | 0.0487 | 0.1987 | |
| Contrastive-ProbePLM=BioBERT2021.10 | 0.0328 | 0.1546 | |
| Contrastive-ProbePLM=BERT2021.10 | 0.0195 | 0.0696 | |
| X-FACTR (Confidence)PLM=BlueBERT2021.10 | 0.0074 | — | |
| Generative PLMsPLM=T5-small2021.10 | 0.007 | 0.0172 | |
| Generative PLMsPLM=SciFive-large2021.10 | 0.0055 | 0.0203 | |
| Generative PLMsPLM=SciFive-base2021.10 | 0.0053 | 0.0202 | |
| X-FACTR (Order+Order)PLM=BlueBERT2021.10 | 0.005 | — | |
| Mask averagePLM=BioBERT2021.10 | 0.0028 | 0.0303 | |
| X-FACTR (Confidence)PLM=BioBERT2021.10 | 0.0017 | — | |
| Generative PLMsPLM=BART-base2021.10 | 0.0016 | 0.0139 | |
| X-FACTR (Order+Order)PLM=BioBERT2021.10 | 0.0011 | — | |
| Generative PLMsPLM=T5-base2021.10 | 0.0006 | 0.0019 | |
| X-FACTR (Order+Order)PLM=BERT2021.10 | 0.0006 | — | |
| Mask averagePLM=BERT2021.10 | 0.0006 | 0.0073 | |
| X-FACTR (Confidence)PLM=BERT2021.10 | 0.0005 | — | |
| Mask averagePLM=BlueBERT2021.10 | 0.0005 | 0.0139 |