Category Retrieval on Geology Amazon (test)
40.69R@50SCIPATTON
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SCIPATTONProtocol=16-shot, Backbone=SciBERT2023.05 | 40.69 | 51.31 | |
| w/o NMLMProtocol=16-shot, Method variant=PATTON ablation2023.05 | 39.29 | 49.59 | |
| PATTONProtocol=16-shot2023.05 | 34.94 | 45.01 | |
| SimCSE.in-domainProtocol=16-shot, Domain=in-domain2023.05 | 33.97 | 44.09 | |
| BERT.MLMProtocol=16-shot, Pre-training=MLM2023.05 | 32.13 | 41.74 | |
| SciBERT.MLMProtocol=16-shot, Backbone=SciBERT, Pre-training=MLM2023.05 | 31.65 | 40.52 | |
| w/o MNPProtocol=16-shot, Method variant=PATTON ablation2023.05 | 31.28 | 40.54 | |
| SPECTERProtocol=16-shot2023.05 | 26.56 | 34.04 | |
| SciBERTProtocol=16-shot, Backbone=SciBERT2023.05 | 23.71 | 30.94 | |
| SimCSE (sup)Protocol=16-shot, Mode=supervised2023.05 | 22.34 | 29.63 | |
| SimCSE (unsup)Protocol=16-shot, Mode=unsupervised2023.05 | 20.45 | 26.82 | |
| BM25Protocol=16-shot2023.05 | 19.02 | 20.92 | |
| GraphFormersProtocol=16-shot2023.05 | 18.92 | 25.94 | |
| BERTProtocol=16-shot2023.05 | 18.82 | 25.94 | |
| LinkBERTProtocol=16-shot2023.05 | 17.14 | 23.05 |