Question Answering on HotpotQA Adversarial
56.8EMCoT-SC + VE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| CoT-SC + VEknowledge=Dataset, Model=GPT-3 davinci-0032023.05 | 56.8 | 25.6 | 60.94 | |
| CoT-SC + VEknowledge=Google, Model=GPT-3 davinci-0032023.05 | 37.7 | 6.5 | 47.98 | |
| CoT-SC + VEknowledge=DRQA, Model=GPT-3 davinci-0032023.05 | 36 | 4.8 | 46.06 | |
| CoT-SC + VEknowledge=Wiki., Model=GPT-3 davinci-0032023.05 | 35.7 | 4.5 | 45.62 | |
| ReAct -> CoT-SCknowledge=Wiki., Model=PaLM2023.05 | 35.1 | 1.7 | — | |
| CoT-SC -> ReActknowledge=Wiki., Model=PaLM2023.05 | 34.2 | 0.8 | — | |
| CoTModel=GPT-3 davinci-0032023.05 | 31.8 | — | 38.3 | |
| CoT-SCModel=GPT-3 davinci-0032023.05 | 31.2 | — | 34.97 | |
| StandardModel=GPT-3 davinci-0032023.05 | 23.1 | — | 43.24 | |
| CoT-SC + Calib.knowledge=Dataset, Model=GPT-3 davinci-0032023.05 | — | — | 49 |