Linguistic Reasoning on BIG-bench Hard Hyperbaton (test)
74Test AccuracyPE2
Evaluation results
| Method | Links | |
|---|---|---|
| PE2Final Prompt=Choose the correct sentence using adjective order: opinion, size, age, shape, color, origin, material, purpose. Note: 'purpose' adjectives, like 'hiking', often come last., Task Model=Mistral-7B-Instruct-v0.2, Prompt Proposal Model=gpt-4-turbo2023.11 | 74 | |
| APOFinal Prompt=Classify adjective order: opinion, size, age, shape, color, origin, material, purpose. For words fitting multiple categories, prioritize purpose, then material. Ignore rare exceptions., Task Model=Mistral-7B-Instruct-v0.2, Prompt Proposal Model=gpt-4-turbo2023.11 | 72 | |
| Zero-shot CoTFinal Prompt=Let's think step by step., Task Model=Mistral-7B-Instruct-v0.2, Prompt Proposal Model=gpt-4-turbo2023.11 | 52 | |
| Iterative APEFinal Prompt=Let's systematically go through every step., Task Model=Mistral-7B-Instruct-v0.2, Prompt Proposal Model=gpt-4-turbo2023.11 | 48 |