# FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT Source: [FAT ALBERT: Finding Answers in Large Texts using Semantic Similarity Attention Layer based on BERT]() Collection index: [llms.txt]() Data updated: 2025-10-25T22:34:00.401Z Authors: Omar Mossad, Amgad Ahmed, Anandharaju Raju, Hari Karthikeyan, Zayed Ahmed Published: 2020-08-22T08:04:21.000Z ## Abstract Machine based text comprehension has always been a significant research field in natural language processing. Once a full understanding of the text context and semantics is achieved, a deep learning model can be trained to solve a large subset of tasks, e.g. text summarization, classification and question answering. In this paper we focus on the question answering problem, specifically the multiple choice type of questions. We develop a model based on BERT, a state-of-the-art transformer network. Moreover, we alleviate the ability of BERT to support large text corpus by extracting the highest influence sentences through a semantic similarity model. Evaluations of our proposed model demonstrate that it outperforms the leading models in the MovieQA challenge and we are currently ranked first in the leader board with test accuracy of 87.79%. Finally, we discuss the model shortcomings and suggest possible improvements to overcome these limitations. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()