Analogy Generation on E-KAR
77.8AccuracyHuman
Evaluation Results
| Method | Links | |
|---|---|---|
| Human2023.05 | 77.8 | |
| InstructGPTversion=003, augmentation=ANALOGYKB2023.05 | 75 | |
| ChatGPTaugmentation=ANALOGYKB2023.05 | 69 | |
| InstructGPTversion=003, augmentation=Human2023.05 | 68 | |
| InstructGPTversion=003, augmentation=ANALOGYKB, subset=same2023.05 | 64 | |
| ChatGPTaugmentation=Human2023.05 | 64 | |
| ChatGPTaugmentation=ANALOGYKB, subset=same2023.05 | 64 | |
| InstructGPTversion=0032023.05 | 61 | |
| ChatGPT2023.05 | 58 | |
| AnalogyT52023.05 | 57 | |
| RoBERTa-Large + ANALOGYKBBackbone=RoBERTa-Large, Knowledge Enhancement=ANALOGYKB, Filtering=Checked2023.05 | 53.43 | |
| DeBERTa-v3 + ANALOGYKBBackbone=DeBERTa-v3, Knowledge Enhancement=ANALOGYKB, Filtering=Checked2023.05 | 53.05 | |
| DeBERTa-v3Backbone=DeBERTa-v32023.05 | 47.18 | |
| RoBERTa-LargeBackbone=RoBERTa-Large2023.05 | 46.7 | |
| RoBERTa-Large + ANALOGYKB (w/o check)Backbone=RoBERTa-Large, Knowledge Enhancement=ANALOGYKB, Filtering=Unchecked2023.05 | 45.34 | |
| DeBERTa-v3 + ANALOGYKB (w/o check)Backbone=DeBERTa-v3, Knowledge Enhancement=ANALOGYKB, Filtering=Unchecked2023.05 | 43.89 | |
| AnalogyT5variant=same2023.05 | 42 | |
| ERNIEBackbone=ERNIE2023.05 | 40.83 | |
| LUKEBackbone=LUKE2023.05 | 40.45 | |
| T5-LargeBackbone=T5-Large2023.05 | 40.08 | |
| BERT-LargeBackbone=BERT-Large2023.05 | 36.64 | |
| Word Embedding from InstructGPTBackbone=InstructGPT, Embedding Type=Word2023.05 | 33.41 | |
| Word Embedding from RoBERTa-LargeBackbone=RoBERTa-Large, Embedding Type=Word2023.05 | 28.2 | |
| Sentence Embedding from SentenceBERTBackbone=SentenceBERT, Embedding Type=Sentence2023.05 | 25.4 | |
| Sentence Embedding from SimCSEBackbone=SimCSE, Embedding Type=Sentence2023.05 | 23.5 | |
| vanilla T52023.05 | 13 |