Question Answering on QASPER Extractive (test)
53.3F1LongT5-large
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LongT5-largeModel size=large2022.09 | 53.3 | — | |
| LongT5-3BModel size=3B2022.09 | 53.1 | — | |
| BART-LS2022.09 | 48.7 | — | |
| LongT5-baseModel size=base2022.09 | 46.6 | — | |
| DOCHOPPERruntime=74.6/s2021.06 | 36.4 | 19.5 | |
| LEDruntime=0.5/s2021.06 | 31 | — | |
| Sequential (ETC)runtime=0.6±0.1/s2021.06 | 27 | 12.4 | |
| Retrieval + ETCretriever=BM25, runtime=8.3/s2021.06 | 19.1 | 9.6 |