Analogy Generation on SAT (test)
91AccuracyChatGPT + ANALOGYKB
Evaluation Results
| Method | Links | |
|---|---|---|
| ChatGPT + ANALOGYKBStrategy=ANALOGYKB retrieved examples2023.05 | 91 | |
| InstructGPT003 + ANALOGYKBStrategy=ANALOGYKB retrieved examples2023.05 | 85 | |
| ChatGPT + ANALOGYKBsameStrategy=ANALOGYKB (same relations) retrieved examples2023.05 | 81 | |
| ChatGPT + HumanStrategy=Human-written examples2023.05 | 80 | |
| ChatGPTStrategy=0-shot2023.05 | 78 | |
| InstructGPT003 + ANALOGYKBsameStrategy=ANALOGYKB (same relations) retrieved examples2023.05 | 77 | |
| InstructGPT003 + HumanStrategy=Human-written examples2023.05 | 74 | |
| AnalogyT5Backbone=T5-Large, Training=Trained on ANALOGYKB2023.05 | 64 | |
| InstructGPT003Strategy=0-shot2023.05 | 60 | |
| AnalogyT5sameBackbone=T5-Large, Training=Trained on ANALOGYKB (same relations)2023.05 | 37 | |
| vanilla T5Backbone=T5-Large, Strategy=Vanilla2023.05 | 8 |