# Transformer-based Language Models for Factoid Question Answering at BioASQ9b

Source: [Transformer-based Language Models for Factoid Question Answering at BioASQ9b](<https://www.sota2.com/research/paper/68fa6ad21e209131cea2a8b5>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T23:34:16.637Z

Authors: Urvashi Khanna, Diego Moll\\'a

Published: 2021-09-15T10:01:06.000Z

## Abstract

In this work, we describe our experiments and participating systems in the BioASQ Task 9b Phase B challenge of biomedical question answering. We have focused on finding the ideal answers and investigated multi-task fine-tuning and gradual unfreezing techniques on transformer-based language models. For factoid questions, our ALBERT-based systems ranked first in test batch 1 and fourth in test batch 2. Our DistilBERT systems outperformed the ALBERT variants in test batches 4 and 5 despite having 81% fewer parameters than ALBERT. However, we observed that gradual unfreezing had no significant impact on the model's accuracy compared to standard fine-tuning.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2109.07185>)

[PDF](<https://arxiv.org/pdf/2109.07185.pdf>)