Numerical Question Answering on FinQA 1.0 (test)
91.16Execution AccuracyHuman Expert Performance
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Human Expert Performance2022.12 | 91.16 | 87.49 | |
| APOLLOProtocol=Fine-tuning, Ensemble=true2022.12 | 71.07 | 68.94 | |
| TabT5Protocol=Fine-tuning, Backbone=T5, Ensemble=true2022.12 | 70.79 | 68 | |
| GPT-4Protocol=Prompting, Mode=zero-shot2022.12 | 68.79 | — | |
| Program-of-ThoughtProtocol=Prompting2022.12 | 68.1 | 67.3 | |
| CellRetriever+UniLMProtocol=Fine-tuning, Backbone=UniLM, Ensemble=true2022.12 | 68 | 65.21 | |
| APOLLOProtocol=Fine-tuning2022.12 | 67.99 | 65.6 | |
| DyRRenProtocol=Fine-tuning, Backbone=ROBERTa-large2022.12 | 63.3 | 61.29 | |
| ELASTICProtocol=Fine-tuning, Backbone=ROBERTa-large2022.12 | 62.16 | 57.54 | |
| FinQANetProtocol=Fine-tuning, Backbone=ROBERTa-large2022.12 | 61.24 | 58.86 | |
| General Crowd Performance2022.12 | 50.68 | 48.17 | |
| Retriever+NeRdProtocol=Fine-tuning, Backbone=ROBERTa-large2022.12 | 48.57 | 46.76 | |
| GPT-3.5-turboProtocol=Prompting, Mode=zero-shot2022.12 | 48.56 | — | |
| LongformerProtocol=Fine-tuning, Backbone=ROBERTa-large2022.12 | 21.9 | 20.48 |