Category Retrieval on Amazon Economics (test)
35.82Recall@50SCIPATTON
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SCIPATTONProtocol=16-shot, Backbone=SciBERT2023.05 | 35.82 | 46.05 | |
| w/o NMLMProtocol=16-shot, Method variant=PATTON ablation2023.05 | 35.17 | 46.07 | |
| PATTONProtocol=16-shot2023.05 | 32.1 | 42.19 | |
| SPECTERProtocol=16-shot2023.05 | 31.26 | 40.79 | |
| SciBERTProtocol=16-shot, Backbone=SciBERT2023.05 | 29.8 | 38.66 | |
| w/o MNPProtocol=16-shot, Method variant=PATTON ablation2023.05 | 29.54 | 39.57 | |
| SciBERT.MLMProtocol=16-shot, Backbone=SciBERT, Pre-training=MLM2023.05 | 29.23 | 39.18 | |
| SimCSE.in-domainProtocol=16-shot, Domain=in-domain2023.05 | 28.44 | 37.81 | |
| SimCSE (sup)Protocol=16-shot, Mode=supervised2023.05 | 28.07 | 36.51 | |
| BERT.MLMProtocol=16-shot, Pre-training=MLM2023.05 | 27.13 | 36 | |
| SimCSE (unsup)Protocol=16-shot, Mode=unsupervised2023.05 | 25.83 | 33.42 | |
| GraphFormersProtocol=16-shot2023.05 | 24.48 | 32.16 | |
| BERTProtocol=16-shot2023.05 | 23.95 | 31.54 | |
| LinkBERTProtocol=16-shot2023.05 | 22.69 | 30.77 | |
| BM25Protocol=16-shot2023.05 | 19.14 | 22.49 |