# BERTVision -- A Parameter-Efficient Approach for Question Answering Source: [BERTVision -- A Parameter-Efficient Approach for Question Answering]() Collection index: [llms.txt]() Data updated: 2025-10-28T13:56:52.057Z Authors: Siduo Jiang, Cristopher Benge, William Casey King Published: 2022-02-24T17:16:25.000Z ## Abstract We present a highly parameter efficient approach for Question Answering that significantly reduces the need for extended BERT fine-tuning. Our method uses information from the hidden state activations of each BERT transformer layer, which is discarded during typical BERT inference. Our best model achieves maximal BERT performance at a fraction of the training time and GPU or TPU expense. Performance is further improved by ensembling our model with BERTs predictions. Furthermore, we find that near optimal performance can be achieved for QA span annotation using less training data. Our experiments show that this approach works well not only for span annotation, but also for classification, suggesting that it may be extensible to a wider range of tasks. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. ## Original sources and implementation [Original paper]() [PDF]()