Question Answering on TriviaQA (dev)
79.5F1 Score (Full)BIGBIRD-ITC
Evaluation results
| Method | Links | |
|---|---|---|
| BIGBIRD-ITCModel Size=Base2020.07 | 79.5 | |
| BIGBIRD-ETCModel Size=Base2020.07 | 78.7 | |
| LongformerModel Size=Base2020.07 | 75.2 | |
| RoBERTaModel Size=Base2020.07 | 74.3 | |
| PMRNumber of training examples=128, Number of Parameters=406M2023.10 | 68.6 | |
| PMRNumber of training examples=64, Number of Parameters=406M2023.10 | 67.1 | |
| PMRNumber of training examples=32, Number of Parameters=406M2023.10 | 66.3 | |
| FewshotQANumber of training examples=64, Number of Parameters=406M2023.10 | 57.9 | |
| FewshotQANumber of training examples=32, Number of Parameters=406M2023.10 | 56.7 | |
| PMRNumber of training examples=16, Number of Parameters=406M2023.10 | 56.2 | |
| FewshotQA w/ MINPROMPTNumber of training examples=64, Number of Parameters=406M2023.10 | 55.3 | |
| FewshotQANumber of training examples=128, Number of Parameters=406M2023.10 | 55.2 | |
| FewshotQA w/ MINPROMPTNumber of training examples=32, Number of Parameters=406M2023.10 | 53.5 | |
| FewshotQA w/ MINPROMPTNumber of training examples=128, Number of Parameters=406M2023.10 | 52.9 | |
| FewshotQA w/ MINPROMPTNumber of training examples=16, Number of Parameters=406M2023.10 | 50.9 | |
| FewshotQANumber of training examples=16, Number of Parameters=406M2023.10 | 47.1 | |
| Splinter w/ MINPROMPTNumber of training examples=128, Number of Parameters=110M2023.10 | 45.4 | |
| SplinterNumber of training examples=128, Number of Parameters=110M2023.10 | 44.7 | |
| Splinter w/ MINPROMPTNumber of training examples=16, Number of Parameters=110M2023.10 | 35.7 | |
| Splinter w/ MINPROMPTNumber of training examples=32, Number of Parameters=110M2023.10 | 35.6 | |
| SplinterNumber of training examples=64, Number of Parameters=110M2023.10 | 35.5 | |
| Splinter w/ MINPROMPTNumber of training examples=64, Number of Parameters=110M2023.10 | 35.4 | |
| SplinterNumber of training examples=32, Number of Parameters=110M2023.10 | 28.9 | |
| SpanBERTNumber of training examples=128, Number of Parameters=110M2023.10 | 26.3 | |
| RoBERTaNumber of training examples=128, Number of Parameters=110M2023.10 | 19.1 | |
| SplinterNumber of training examples=16, Number of Parameters=110M2023.10 | 18.9 | |
| SpanBERTNumber of training examples=64, Number of Parameters=110M2023.10 | 15.9 | |
| SpanBERTNumber of training examples=32, Number of Parameters=110M2023.10 | 15.1 | |
| RoBERTaNumber of training examples=64, Number of Parameters=110M2023.10 | 12.5 | |
| SpanBERTNumber of training examples=16, Number of Parameters=110M2023.10 | 11.6 | |
| RoBERTaNumber of training examples=32, Number of Parameters=110M2023.10 | 10.5 | |
| RoBERTaNumber of training examples=16, Number of Parameters=110M2023.10 | 4.4 |