Date Understanding on BIG-bench Hard (test)
75.2Test AccuracyDLN-2
Evaluation results
| Method | Links | |
|---|---|---|
| DLN-2LLM=GPT-32023.06 | 75.2 | |
| CoTLLM=GPT-32023.06 | 72.4 | |
| 0-shotLLM=GPT-32023.06 | 56.4 | |
| PE2Final Prompt=Let's think step by step. For date questions, calculate from the given date, then choose the closest matching option. Ignore irrelevant info., Task Model=Mistral-7B-Instruct-v0.2, Prompt Proposal Model=gpt-4-turbo2023.11 | 56 | |
| DLN-1LLM=GPT-32023.06 | 55.7 | |
| PE2Final Prompt=Analyzing the given information, let's calculate the solution. Remember to consider the context provided, such as references to 'today' or specific dates., Task Model=gpt-3.5-turbo-instruct, Prompt Proposal Model=gpt-42023.11 | 54.4 | |
| Iterative APEFinal Prompt=Let's move forward by dividing it into manageable steps., Task Model=Mistral-7B-Instruct-v0.2, Prompt Proposal Model=gpt-4-turbo2023.11 | 48 | |
| APOFinal Prompt=Accurately calculate dates from given inputs, considering the current reference point (e.g., 'yesterday', 'today') and applying correct calendar arithmetic, including month lengths and leap years. Ignore irrelevant details and assumptions beyond provided information., Task Model=Mistral-7B-Instruct-v0.2, Prompt Proposal Model=gpt-4-turbo2023.11 | 48 | |
| Iterative APEFinal Prompt=Let's dissect it and ponder over each phase., Task Model=gpt-3.5-turbo-instruct, Prompt Proposal Model=gpt-42023.11 | 46.7 | |
| APOFinal Prompt=Determine the exact date from the scenario, considering cultural date formats, time zones, and periods. Use the provided date as a reference. Account for leading zeros, leap years, relative dates, and event-based time references. Provide the result in MM/DD/YYYY format., Task Model=gpt-3.5-turbo-instruct, Prompt Proposal Model=gpt-42023.11 | 45 | |
| Zero-shot CoTFinal Prompt=Let's think step by step., Task Model=gpt-3.5-turbo-instruct, Prompt Proposal Model=gpt-42023.11 | 39.1 | |
| Zero-shot CoTFinal Prompt=Let's think step by step., Task Model=Mistral-7B-Instruct-v0.2, Prompt Proposal Model=gpt-4-turbo2023.11 | 36 | |
| APELLM=GPT-32023.06 | 32.1 | |
| APE-400LLM=GPT-32023.06 | 23.5 |