Multi-hop Reasoning on CommaQA-E (test)
70Exact MatchChatGPT (SKiC)
Evaluation Results
| Method | Links | |
|---|---|---|
| ChatGPT (SKiC)Model=ChatGPT, Prompting=SKiC†, #-shots=22023.08 | 70 | |
| text-davinci-003 (SKiC)Model=text-davinci-003, Prompting=SKiC†, #-shots=22023.08 | 66 | |
| ChatGPT (Decomp)Model=ChatGPT, Prompting=Decomp, #-shots=122023.08 | 64 | |
| text-davinci-003 (Decomp)Model=text-davinci-003, Prompting=Decomp, #-shots=122023.08 | 58 | |
| ChatGPT (CoT)Model=ChatGPT, Prompting=CoT, #-shots=42023.08 | 55 | |
| ChatGPT (4-shots)Model=ChatGPT, Prompting=4-shots, #-shots=42023.08 | 47 | |
| LLAMA-65B (SKiC)Model=LLAMA-65B, Prompting=SKiC†, #-shots=22023.08 | 44 | |
| text-davinci-003 (CoT)Model=text-davinci-003, Prompting=CoT, #-shots=42023.08 | 44 | |
| text-davinci-003 (4-shots)Model=text-davinci-003, Prompting=4-shots, #-shots=42023.08 | 42 | |
| ChatGPT (zero-shot)Model=ChatGPT, Prompting=zero-shot, #-shots=02023.08 | 42 | |
| text-davinci-003 (zero-shot)Model=text-davinci-003, Prompting=zero-shot, #-shots=02023.08 | 34 | |
| LLAMA-65B (Decomp)Model=LLAMA-65B, Prompting=Decomp, #-shots=122023.08 | 32 | |
| LLAMA-65B (CoT)Model=LLAMA-65B, Prompting=CoT, #-shots=42023.08 | 27 | |
| LLAMA-65B (4-shots)Model=LLAMA-65B, Prompting=4-shots, #-shots=42023.08 | 15 | |
| LLAMA-65B (zero-shot)Model=LLAMA-65B, Prompting=zero-shot, #-shots=02023.08 | 12 |