Financial Question Answering on TATQA 1.0 (test)
89.99AccuracyProgram-of-Thoughts (PoT)
Evaluation results
| Method | Links | |
|---|---|---|
| Program-of-Thoughts (PoT)Backbone=GPT-42024.02 | 89.99 | |
| EEDPBackbone=GPT-42024.02 | 88.67 | |
| Chain-of-Thought (CoT)Backbone=GPT-42024.02 | 86.91 | |
| Program-of-Thoughts (PoT)Backbone=GPT-3.5-Turbo2024.02 | 82.11 | |
| EEDPBackbone=PaLM 2-540B2024.02 | 81.51 | |
| EEDPBackbone=GPT-3.5-Turbo2024.02 | 79.73 | |
| Chain-of-Thought (CoT)Backbone=GPT-3.5-Turbo2024.02 | 77.57 | |
| Chain-of-Thought (CoT)Backbone=PaLM 2-540B2024.02 | 62.93 | |
| Program-of-Thoughts (PoT)Backbone=PaLM 2-540B2024.02 | 61.6 | |
| Chain-of-Thought (CoT)Backbone=Mistral-7B2024.02 | 59.14 | |
| DecomposersBackbone=PaLM 2-540B2024.02 | 57.94 | |
| Chain-of-Thought (CoT)Backbone=MAmmoTH-13B2024.02 | 56.25 | |
| EEDPBackbone=Mistral-7B2024.02 | 56.06 | |
| DirectBackbone=GPT-42024.02 | 55.81 | |
| DecomposersBackbone=GPT-42024.02 | 47.46 | |
| DirectBackbone=PaLM 2-540B2024.02 | 44.66 | |
| EEDPBackbone=Llama 2-13B2024.02 | 40.95 | |
| Chain-of-Thought (CoT)Backbone=Llama 2-13B2024.02 | 35.95 | |
| Program-of-Thoughts (PoT)Backbone=Llama 2-13B2024.02 | 34.16 | |
| DirectBackbone=GPT-3.5-Turbo2024.02 | 31.38 | |
| DecomposersBackbone=GPT-3.5-Turbo2024.02 | 28.53 | |
| DecomposersBackbone=Llama 2-13B2024.02 | 25.93 | |
| DecomposersBackbone=MAmmoTH-13B2024.02 | 22.89 | |
| DirectBackbone=MAmmoTH-13B2024.02 | 19.11 | |
| Program-of-Thoughts (PoT)Backbone=Mistral-7B2024.02 | 16.53 | |
| DirectBackbone=Mistral-7B2024.02 | 10.92 | |
| Program-of-Thoughts (PoT)Backbone=MAmmoTH-13B2024.02 | 10.02 | |
| DecomposersBackbone=Mistral-7B2024.02 | 7.24 | |
| EEDPBackbone=MAmmoTH-13B2024.02 | 4.37 | |
| DirectBackbone=Llama 2-13B2024.02 | 3.36 |