Financial Question Answering on FinanceIQ
73.6Accuracy (%)Re-TASK (Full)
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Re-TASK (Full)Prompt Category=Re-TASK, Backbone=Yi-1.5-9B2024.08 | 73.6 | — | |
| Step-BackPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.08 | 66.85 | — | |
| Plan-and-SolvePrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.08 | 66.29 | — | |
| Re-TASK (+K0)Prompt Category=Re-TASK, Backbone=Yi-1.5-9B2024.08 | 66.29 | — | |
| One-shot CoTPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.08 | 64.33 | — | |
| Three-shot CoTPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.08 | 63.82 | — | |
| Zero-shot CoT + SCPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.08 | 61.8 | — | |
| Re-TASK (Lite)Prompt Category=Re-TASK, Backbone=Yi-1.5-9B2024.08 | 61.8 | — | |
| Zero-shot CoTPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.08 | 53.93 | — | |
| Re-TASK (Full)Prompt Category=Re-TASK, Backbone=Llama3-Chinese-8B2024.08 | 52.81 | — | |
| Re-TASK (Full)Prompt Category=Re-TASK, Backbone=Qwen1.5-7B2024.08 | 51.69 | — | |
| Re-TASK (Lite)Prompt Category=Re-TASK, Backbone=Qwen1.5-7B2024.08 | 49.44 | — | |
| Zero-shot CoT + SCPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.08 | 46.63 | — | |
| One-shot CoTPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.08 | 46.07 | — | |
| Three-shot CoTPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.08 | 46.07 | — | |
| Step-BackPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.08 | 44.38 | — | |
| Zero-shot CoTPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.08 | 43.82 | — | |
| Plan-and-SolvePrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.08 | 41.01 | — | |
| Re-TASK (+K0)Prompt Category=Re-TASK, Backbone=Qwen1.5-7B2024.08 | 39.89 | — | |
| Re-TASK (Lite)Prompt Category=Re-TASK, Backbone=Llama3-Chinese-8B2024.08 | 38.2 | — | |
| Zero-shot CoTPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.08 | 36.52 | — | |
| One-shot CoTPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.08 | 34.69 | — | |
| Zero-shot CoT + SCPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.08 | 34.27 | — | |
| Three-shot CoTPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.08 | 34.27 | — | |
| Re-TASK (+K0)Prompt Category=Re-TASK, Backbone=Llama3-Chinese-8B2024.08 | 34.27 | — | |
| Step-BackPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.08 | 30.9 | — | |
| Plan-and-SolvePrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.08 | 30.34 | — | |
| One-shot CoTPrompt Category=Traditional CoT, Backbone=Average Across Models2024.08 | — | 3.6 | |
| Plan-and-SolvePrompt Category=Traditional CoT, Backbone=Average Across Models2024.08 | — | 1.12 | |
| Re-TASK (+K0)Prompt Category=Re-TASK, Backbone=Average Across Models2024.08 | — | 2.06 | |
| Re-TASK (Full)Prompt Category=Re-TASK, Backbone=Average Across Models2024.08 | — | 14.61 | |
| Re-TASK (Lite)Prompt Category=Re-TASK, Backbone=Average Across Models2024.08 | — | 5.06 | |
| Step-BackPrompt Category=Traditional CoT, Backbone=Average Across Models2024.08 | — | 2.62 | |
| Three-shot CoTPrompt Category=Traditional CoT, Backbone=Average Across Models2024.08 | — | 3.3 | |
| Zero-shot CoT + SCPrompt Category=Traditional CoT, Backbone=Average Across Models2024.08 | — | 2.81 |