Category Retrieval on Mathematics Amazon (test)
31.4R@50SCIPATTON
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| SCIPATTONProtocol=16-shot, Backbone=SciBERT2023.05 | 31.4 | 40.38 | |
| w/o NMLMProtocol=16-shot, Method variant=PATTON ablation2023.05 | 30.85 | 39.89 | |
| PATTONProtocol=16-shot2023.05 | 27.44 | 34.97 | |
| SciBERTProtocol=16-shot, Backbone=SciBERT2023.05 | 24.7 | 33.55 | |
| SimCSE.in-domainProtocol=16-shot, Domain=in-domain2023.05 | 24.54 | 31.66 | |
| SPECTERProtocol=16-shot2023.05 | 23.86 | 31.11 | |
| w/o MNPProtocol=16-shot, Method variant=PATTON ablation2023.05 | 22.47 | 30.2 | |
| BM25Protocol=16-shot2023.05 | 20.76 | 24.55 | |
| BERT.MLMProtocol=16-shot, Pre-training=MLM2023.05 | 20.69 | 27.17 | |
| SciBERT.MLMProtocol=16-shot, Backbone=SciBERT, Pre-training=MLM2023.05 | 20.65 | 27.67 | |
| SimCSE (sup)Protocol=16-shot, Mode=supervised2023.05 | 20.29 | 26.23 | |
| SimCSE (unsup)Protocol=16-shot, Mode=unsupervised2023.05 | 17.91 | 23.19 | |
| LinkBERTProtocol=16-shot2023.05 | 17.25 | 23.21 | |
| BERTProtocol=16-shot2023.05 | 16.73 | 22.66 | |
| GraphFormersProtocol=16-shot2023.05 | 16.65 | 22.41 |