Question Answering on BioASQ factoid 7b (test)
47.4SAccBioMegatron
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| BioMegatronParameters=1.2b, Vocabulary=BERT-uncased, Pre-training source=Fine-tuned from general domain model checkpoint, fine-tuning epochs=30, pre-finetuning dataset=SQuAD, num-fc-layers=2, fc-hidden-size=2048, fc-dropout=0.1, max-seq-length=512, learning-rate=3e-5, optimizer=Adam2020.10 | 47.4 | 60.9 | 52.4 | |
| BioMegatronParameters=345m, Vocabulary=BERT-uncased, Pre-training source=PubMed (from scratch), fine-tuning epochs=30, pre-finetuning dataset=SQuAD, num-fc-layers=2, fc-hidden-size=2048, fc-dropout=0.1, max-seq-length=512, learning-rate=3e-5, optimizer=Adam2020.10 | 46.2 | 62.6 | 52.5 | |
| BioMegatronParameters=800m, Vocabulary=BERT-uncased, Pre-training source=PubMed (from scratch), fine-tuning epochs=30, pre-finetuning dataset=SQuAD, num-fc-layers=2, fc-hidden-size=2048, fc-dropout=0.1, max-seq-length=512, learning-rate=3e-5, optimizer=Adam2020.10 | 45.2 | 58.6 | 50.4 | |
| QA1Challenge Rank=12021.04 | 44.83 | 58.62 | 51.15 | |
| UNCC_QA_1Challenge Rank=22021.04 | 44.83 | 58.62 | 51.15 | |
| ELECTRAMedChallenge Rank=12021.04 | 44.62 | 51.28 | 47.95 | |
| BioBERT-LargeParameters=345m, Vocabulary=BERT-cased, fine-tuning epochs=30, pre-finetuning dataset=SQuAD, num-fc-layers=2, fc-hidden-size=2048, fc-dropout=0.1, max-seq-length=512, learning-rate=3e-5, optimizer=Adam2020.10 | 42.8 | 62.8 | 50.1 | |
| google-gold-inputChallenge Rank=32021.04 | 41.38 | 65.52 | 50.23 | |
| KU-DMIS-1Challenge Rank=22021.04 | 41.03 | 53.85 | 46.37 | |
| ELECTRAMedChallenge Rank=72021.04 | 37.93 | 58.62 | 46.62 | |
| BioBERT-BaseParameters=110m, Vocabulary=BERT-cased, fine-tuning epochs=30, pre-finetuning dataset=SQuAD, num-fc-layers=2, fc-hidden-size=2048, fc-dropout=0.1, max-seq-length=512, learning-rate=3e-5, optimizer=Adam2020.10 | 30.8 | 64.1 | 41.1 | |
| BJUTNLPGroupChallenge Rank=32021.04 | 30.77 | 41.03 | 34.83 | |
| auth-qa-1Challenge Rank=42021.04 | 25.64 | 30.77 | 27.78 |