# LXMERT Model Compression for Visual Question Answering Source: [LXMERT Model Compression for Visual Question Answering]() Collection index: [llms.txt]() Data updated: 2026-01-02T03:49:59.095Z Authors: Maryam Hashemi, Ghazaleh Mahmoudi, Sara Kodeiri, Hadi Sheikhi, Sauleh Eetemadi Published: 2023-10-23T19:46:41.000Z ## Abstract Large-scale pretrained models such as LXMERT are becoming popular for learning cross-modal representations on text-image pairs for vision-language tasks. According to the lottery ticket hypothesis, NLP and computer vision models contain smaller subnetworks capable of being trained in isolation to full performance. In this paper, we combine these observations to evaluate whether such trainable subnetworks exist in LXMERT when fine-tuned on the VQA task. In addition, we perform a model size cost-benefit analysis by investigating how much pruning can be done without significant loss in accuracy. Our experiment results demonstrate that LXMERT can be effectively pruned by 40%-60% in size with 3% loss in accuracy. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Visual Question Answering on VQA v2 (test-dev)](): Visual Question Answering; VQA v2 (test-dev); Overall Accuracy: not recorded. - [Visual Question Answering on VQA v2 (test-std)](): Visual Question Answering; VQA v2 (test-std); Accuracy: not recorded. ## Original sources and implementation [Original paper]() [PDF]()