List your company
ResearchFormerly WizWand

Financial Question Answering on FinanceIQ

73.6Accuracy (%)Re-TASK (Full)
28.609640.289851.9763.6502Aug 13, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Re-TASK (Full)Prompt Category=Re-TASK, Backbone=Yi-1.5-9B2024.0873.6—
Step-BackPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.0866.85—
Plan-and-SolvePrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.0866.29—
Re-TASK (+K0)Prompt Category=Re-TASK, Backbone=Yi-1.5-9B2024.0866.29—
One-shot CoTPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.0864.33—
Three-shot CoTPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.0863.82—
Zero-shot CoT + SCPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.0861.8—
Re-TASK (Lite)Prompt Category=Re-TASK, Backbone=Yi-1.5-9B2024.0861.8—
Zero-shot CoTPrompt Category=Traditional CoT, Backbone=Yi-1.5-9B2024.0853.93—
Re-TASK (Full)Prompt Category=Re-TASK, Backbone=Llama3-Chinese-8B2024.0852.81—
Re-TASK (Full)Prompt Category=Re-TASK, Backbone=Qwen1.5-7B2024.0851.69—
Re-TASK (Lite)Prompt Category=Re-TASK, Backbone=Qwen1.5-7B2024.0849.44—
Zero-shot CoT + SCPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.0846.63—
One-shot CoTPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.0846.07—
Three-shot CoTPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.0846.07—
Step-BackPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.0844.38—
Zero-shot CoTPrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.0843.82—
Plan-and-SolvePrompt Category=Traditional CoT, Backbone=Qwen1.5-7B2024.0841.01—
Re-TASK (+K0)Prompt Category=Re-TASK, Backbone=Qwen1.5-7B2024.0839.89—
Re-TASK (Lite)Prompt Category=Re-TASK, Backbone=Llama3-Chinese-8B2024.0838.2—
Zero-shot CoTPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.0836.52—
One-shot CoTPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.0834.69—
Zero-shot CoT + SCPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.0834.27—
Three-shot CoTPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.0834.27—
Re-TASK (+K0)Prompt Category=Re-TASK, Backbone=Llama3-Chinese-8B2024.0834.27—
Step-BackPrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.0830.9—
Plan-and-SolvePrompt Category=Traditional CoT, Backbone=Llama3-Chinese-8B2024.0830.34—
One-shot CoTPrompt Category=Traditional CoT, Backbone=Average Across Models2024.08—3.6
Plan-and-SolvePrompt Category=Traditional CoT, Backbone=Average Across Models2024.08—1.12
Re-TASK (+K0)Prompt Category=Re-TASK, Backbone=Average Across Models2024.08—2.06
Re-TASK (Full)Prompt Category=Re-TASK, Backbone=Average Across Models2024.08—14.61
Re-TASK (Lite)Prompt Category=Re-TASK, Backbone=Average Across Models2024.08—5.06
Step-BackPrompt Category=Traditional CoT, Backbone=Average Across Models2024.08—2.62
Three-shot CoTPrompt Category=Traditional CoT, Backbone=Average Across Models2024.08—3.3
Zero-shot CoT + SCPrompt Category=Traditional CoT, Backbone=Average Across Models2024.08—2.81