Conversational Financial Question Answering on ConvFinQA (test)
78.81AccuracyProgram-of-Thoughts (PoT)
Evaluation results
| Method | Links | |
|---|---|---|
| Program-of-Thoughts (PoT)Backbone=GPT-42024.02 | 78.81 | |
| EEDPBackbone=GPT-42024.02 | 77.91 | |
| Chain-of-Thought (CoT)Backbone=GPT-42024.02 | 71.19 | |
| EEDPBackbone=PaLM 2-540B2024.02 | 63.42 | |
| DirectBackbone=GPT-42024.02 | 63.1 | |
| EEDPBackbone=GPT-3.5-Turbo2024.02 | 61.75 | |
| Program-of-Thoughts (PoT)Backbone=GPT-3.5-Turbo2024.02 | 61.19 | |
| Chain-of-Thought (CoT)Backbone=Mistral-7B2024.02 | 48.45 | |
| Chain-of-Thought (CoT)Backbone=GPT-3.5-Turbo2024.02 | 48.33 | |
| Chain-of-Thought (CoT)Backbone=MAmmoTH-13B2024.02 | 46.08 | |
| Program-of-Thoughts (PoT)Backbone=PaLM 2-540B2024.02 | 40.14 | |
| EEDPBackbone=Llama 2-13B2024.02 | 39.42 | |
| Chain-of-Thought (CoT)Backbone=PaLM 2-540B2024.02 | 38 | |
| DirectBackbone=GPT-3.5-Turbo2024.02 | 37.62 | |
| EEDPBackbone=Mistral-7B2024.02 | 36.57 | |
| EEDPBackbone=MAmmoTH-13B2024.02 | 32.77 | |
| Program-of-Thoughts (PoT)Backbone=Llama 2-13B2024.02 | 29.92 | |
| Chain-of-Thought (CoT)Backbone=Llama 2-13B2024.02 | 29.45 | |
| DecomposersBackbone=PaLM 2-540B2024.02 | 22.32 | |
| DirectBackbone=MAmmoTH-13B2024.02 | 21.61 | |
| DirectBackbone=PaLM 2-540B2024.02 | 20.19 | |
| DecomposersBackbone=GPT-42024.02 | 18.76 | |
| Program-of-Thoughts (PoT)Backbone=Mistral-7B2024.02 | 14.48 | |
| DirectBackbone=Mistral-7B2024.02 | 12.35 | |
| DecomposersBackbone=Mistral-7B2024.02 | 11.16 | |
| DecomposersBackbone=GPT-3.5-Turbo2024.02 | 10.5 | |
| DecomposersBackbone=Llama 2-13B2024.02 | 10.35 | |
| Program-of-Thoughts (PoT)Backbone=MAmmoTH-13B2024.02 | 8.78 | |
| DecomposersBackbone=MAmmoTH-13B2024.02 | 7.83 | |
| DirectBackbone=Llama 2-13B2024.02 | 3.8 |