# Multi-Document Financial Question Answering using LLMs Source: [Multi-Document Financial Question Answering using LLMs]() Collection index: [llms.txt]() Data updated: 2025-10-26T03:40:43.406Z Authors: Shalin Shah, Srikanth Ryali, Ramasubbu Venkatesh Published: 2024-11-08T21:03:54.000Z ## Abstract We propose two new methods for multi-document financial question answering. First, a method that uses semantic tagging, and then, queries the index to get the context (RAG\_SEM). And second, a Knowledge Graph (KG\_RAG) based method that uses semantic tagging, and, retrieves knowledge graph triples from a graph database, as context. KG\_RAG uses knowledge graphs constructed using a small model that is fine-tuned using knowledge distillation using a large teacher model. The data consists of 18 10K reports of Apple, Microsoft, Alphabet, NVIDIA, Amazon and Tesla for the years 2021, 2022 and 2023. The list of questions in the data consists of 111 complex questions including many esoteric questions that are difficult to answer and the answers are not completely obvious. As evaluation metrics, we use overall scores as well as segmented scores for measurement including the faithfulness, relevance, correctness, similarity, an LLM based overall score and the rouge scores as well as a similarity of embeddings. We find that both methods outperform plain RAG significantly. KG\_RAG outperforms RAG\_SEM in four out of nine metrics. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()