Analogy Generation on E-KAR (test)
75AccuracyInstructGPT003 + ANALOGYKB
Evaluation Results
| Method | Links | |
|---|---|---|
| InstructGPT003 + ANALOGYKBStrategy=ANALOGYKB retrieved examples2023.05 | 75 | |
| ChatGPT + ANALOGYKBStrategy=ANALOGYKB retrieved examples2023.05 | 69 | |
| InstructGPT003 + HumanStrategy=Human-written examples2023.05 | 68 | |
| InstructGPT003 + ANALOGYKBsameStrategy=ANALOGYKB (same relations) retrieved examples2023.05 | 64 | |
| ChatGPT + HumanStrategy=Human-written examples2023.05 | 64 | |
| ChatGPT + ANALOGYKBsameStrategy=ANALOGYKB (same relations) retrieved examples2023.05 | 64 | |
| InstructGPT003Strategy=0-shot2023.05 | 61 | |
| ChatGPTStrategy=0-shot2023.05 | 58 | |
| AnalogyT5Backbone=T5-Large, Training=Trained on ANALOGYKB2023.05 | 57 | |
| AnalogyT5sameBackbone=T5-Large, Training=Trained on ANALOGYKB (same relations)2023.05 | 42 | |
| vanilla T5Backbone=T5-Large, Strategy=Vanilla2023.05 | 13 |