List your company
ResearchFormerly WizWand

Numerical Question Answering on FinQA (test)

91.16Execution AccuracyHuman Expert Performance
-3.334421.197845.7370.2622Sep 1, 2021May 22, 2022Feb 9, 2023Oct 31, 2023Jul 20, 2024Apr 9, 2025Dec 29, 2025
Updated 8mo ago

Evaluation results

MethodLinks
Human Expert Performance2021.0991.1687.49
Human Expert2022.1091.1687.49
FinQANet-Gold (RoBERTa-large)Input Source=Gold Retriever, Program Generator Backbone=RoBERTa-large2021.097068.76
Multi-Retriever RAG (SecBERT + DPR-FAISS) + Gemini-1.5-pro-latest Generator + Few Shot PromptRetriever Type=Internal + External (Multi-Retriever), Prompting Strategy=Few Shot Prompt2025.1269.37—
ELASTICEncoder=RoBERTa-large2022.1068.9665.21
SecBERT Internal Retriever + Gemini-1.5-pro-latest Generator + Few Shot PromptRetriever Type=Internal, Prompting Strategy=Few Shot Prompt2025.1266.02—
FinQANetEncoder=RoBERTa-large2022.1065.0563.52
ELASTICEncoder=RoBERTa-base2022.1062.6659.28
FinQANet (RoBERTa-large)Retriever=BERT-base, Program Generator Backbone=RoBERTa-large2021.0961.2458.86
FinQANetEncoder=RoBERTa-base2022.1060.158.38
FinQANet (RoBERTa-base)Retriever=BERT-base, Program Generator Backbone=RoBERTa-base2021.0956.154.38
FinQANet (BERT-large)Retriever=BERT-base, Program Generator Backbone=BERT-large2021.0953.5251.62
NeRd2022.1052.4849.9
General Crowd Performance2021.0950.6848.17
Human Non-Expert2022.1050.6848.17
FinQANet (FinBert)Retriever=BERT-base, Program Generator Backbone=FinBert2021.0950.147.52
FinQANet (BERT-base)Retriever=BERT-base, Program Generator Backbone=BERT-base2021.095048
Retriever + NeRd (BERT-base)Retriever=BERT-base, Program Generator=NeRd, Backbone=BERT-base2021.0948.5746.76
Multi-Retriever RAG (SecBERT + DPR-FAISS) + Gemini-1.5-pro-latest GeneratorRetriever Type=Internal + External (Multi-Retriever), Prompting Strategy=Single Prompt2025.1245.33—
SecBERT Internal Retriever + Gemini-1.5-pro-latest GeneratorRetriever Type=Internal, Prompting Strategy=Single Prompt2025.1241.84—
Multi-Retriever RAG (SecBERT + DPR-FAISS) + Gemini-1.0-pro GeneratorRetriever Type=Internal + External (Multi-Retriever), Prompting Strategy=Single Prompt2025.1241.75—
SecBERT Internal Retriever + Gemini-1.0-pro GeneratorRetriever Type=Internal, Prompting Strategy=Single Prompt2025.1239.3—
Gemini-1.5-pro-latestRetriever Type=None, Prompting Strategy=Zero-shot2025.1236.27—
Gemini-1.0-proRetriever Type=None, Prompting Strategy=Zero-shot2025.1234.76—
Multi-Retriever RAG (SecBERT + DPR-FAISS) + Gemini-1.0-pro Generator + Few Shot PromptRetriever Type=Internal + External (Multi-Retriever), Prompting Strategy=Few Shot Prompt2025.1224.69—
SecBERT Internal Retriever + Gemini-1.0-pro Generator + Few Shot PromptRetriever Type=Internal, Prompting Strategy=Few Shot Prompt2025.1222.03—
Pre-Trained Longformer (base)Backbone=Longformer (base), Retriever=None (End-to-End)2021.0921.920.48
Retriever + Seq2seqRetriever=BERT-base, Program Generator=Seq2seq2021.0919.7118.38
NumNet+2022.1010.29—
NumNet2022.102.32—
TF-IDF + Single OpRetriever=TF-IDF, Program Generator=Single Op2021.091.010.9
Graph2Tree2022.100.370
Retriever + Direct GenerationRetriever=BERT-base, Program Generator=Direct Generation2021.090.3—