List your company
ResearchFormerly WizWand

Analogy Generation on E-KAR

77.8AccuracyHuman
10.40827.90445.462.896May 10, 2023
Updated 8mo ago

Evaluation results

MethodLinks
Human2023.0577.8
InstructGPTversion=003, augmentation=ANALOGYKB2023.0575
ChatGPTaugmentation=ANALOGYKB2023.0569
InstructGPTversion=003, augmentation=Human2023.0568
InstructGPTversion=003, augmentation=ANALOGYKB, subset=same2023.0564
ChatGPTaugmentation=Human2023.0564
ChatGPTaugmentation=ANALOGYKB, subset=same2023.0564
InstructGPTversion=0032023.0561
ChatGPT2023.0558
AnalogyT52023.0557
RoBERTa-Large + ANALOGYKBBackbone=RoBERTa-Large, Knowledge Enhancement=ANALOGYKB, Filtering=Checked2023.0553.43
DeBERTa-v3 + ANALOGYKBBackbone=DeBERTa-v3, Knowledge Enhancement=ANALOGYKB, Filtering=Checked2023.0553.05
DeBERTa-v3Backbone=DeBERTa-v32023.0547.18
RoBERTa-LargeBackbone=RoBERTa-Large2023.0546.7
RoBERTa-Large + ANALOGYKB (w/o check)Backbone=RoBERTa-Large, Knowledge Enhancement=ANALOGYKB, Filtering=Unchecked2023.0545.34
DeBERTa-v3 + ANALOGYKB (w/o check)Backbone=DeBERTa-v3, Knowledge Enhancement=ANALOGYKB, Filtering=Unchecked2023.0543.89
AnalogyT5variant=same2023.0542
ERNIEBackbone=ERNIE2023.0540.83
LUKEBackbone=LUKE2023.0540.45
T5-LargeBackbone=T5-Large2023.0540.08
BERT-LargeBackbone=BERT-Large2023.0536.64
Word Embedding from InstructGPTBackbone=InstructGPT, Embedding Type=Word2023.0533.41
Word Embedding from RoBERTa-LargeBackbone=RoBERTa-Large, Embedding Type=Word2023.0528.2
Sentence Embedding from SentenceBERTBackbone=SentenceBERT, Embedding Type=Sentence2023.0525.4
Sentence Embedding from SimCSEBackbone=SimCSE, Embedding Type=Sentence2023.0523.5
vanilla T52023.0513