Question Answering on SearchQA (dev)
68.5F1 (N-gram)PMR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PMRNumber of training examples=128, Number of Parameters=406M2023.10 | 68.5 | — | — | — | |
| FewshotQA w/ MINPROMPTNumber of training examples=128, Number of Parameters=406M2023.10 | 68.1 | — | — | — | |
| FewshotQA w/ MINPROMPTNumber of training examples=64, Number of Parameters=406M2023.10 | 67.1 | — | — | — | |
| FewshotQANumber of training examples=128, Number of Parameters=406M2023.10 | 67 | — | — | — | |
| PMRNumber of training examples=64, Number of Parameters=406M2023.10 | 66.2 | — | — | — | |
| FewshotQANumber of training examples=64, Number of Parameters=406M2023.10 | 65.4 | — | — | — | |
| PMRNumber of training examples=32, Number of Parameters=406M2023.10 | 64.8 | — | — | — | |
| FewshotQANumber of training examples=32, Number of Parameters=406M2023.10 | 61.4 | — | — | — | |
| FewshotQA w/ MINPROMPTNumber of training examples=32, Number of Parameters=406M2023.10 | 60.3 | — | — | — | |
| PMRNumber of training examples=16, Number of Parameters=406M2023.10 | 58.2 | — | — | — | |
| FewshotQA w/ MINPROMPTNumber of training examples=16, Number of Parameters=406M2023.10 | 55.4 | — | — | — | |
| FewshotQANumber of training examples=16, Number of Parameters=406M2023.10 | 54.3 | — | — | — | |
| Splinter w/ MINPROMPTNumber of training examples=64, Number of Parameters=110M2023.10 | 48.6 | — | — | — | |
| Splinter w/ MINPROMPTNumber of training examples=128, Number of Parameters=110M2023.10 | 48.5 | — | — | — | |
| SplinterNumber of training examples=128, Number of Parameters=110M2023.10 | 47.2 | — | — | — | |
| SplinterNumber of training examples=64, Number of Parameters=110M2023.10 | 44.3 | — | — | — | |
| Splinter w/ MINPROMPTNumber of training examples=32, Number of Parameters=110M2023.10 | 39.2 | — | — | — | |
| Splinter w/ MINPROMPTNumber of training examples=16, Number of Parameters=110M2023.10 | 35.2 | — | — | — | |
| SplinterNumber of training examples=32, Number of Parameters=110M2023.10 | 34.8 | — | — | — | |
| RoBERTaNumber of training examples=128, Number of Parameters=110M2023.10 | 27.8 | — | — | — | |
| SplinterNumber of training examples=16, Number of Parameters=110M2023.10 | 26.3 | — | — | — | |
| SpanBERTNumber of training examples=128, Number of Parameters=110M2023.10 | 26.3 | — | — | — | |
| RoBERTaNumber of training examples=64, Number of Parameters=110M2023.10 | 19.8 | — | — | — | |
| SpanBERTNumber of training examples=64, Number of Parameters=110M2023.10 | 18 | — | — | — | |
| SpanBERTNumber of training examples=32, Number of Parameters=110M2023.10 | 14.6 | — | — | — | |
| RoBERTaNumber of training examples=32, Number of Parameters=110M2023.10 | 13.5 | — | — | — | |
| SpanBERTNumber of training examples=16, Number of Parameters=110M2023.10 | 13.3 | — | — | — | |
| RoBERTaNumber of training examples=16, Number of Parameters=110M2023.10 | 6.9 | — | — | — | |
| AMANDA2018.01 | — | 0.486 | 57.7 | — | |
| AMANDATime=≈8* min2018.03 | — | 0.486 | 57.7 | — | |
| AMANDA2018.11 | — | 0.486 | 57.7 | — | |
| AQA2018.11 | — | — | 0.474 | 0.405 | |
| ASR2018.01 | — | 0.439 | 24.2 | — | |
| ASR2018.03 | — | 0.439 | 24.2 | — | |
| ASR2018.11 | — | 0.439 | 24.2 | — | |
| Bi-Attention (150d BiLSTM)Time=≈7 min2018.03 | — | 0.4 | 51.3 | — | |
| Bi-Attention (200d Hybrid MRU-LSTM)Time=≈7 min2018.03 | — | 0.505 | 59.9 | — | |
| Bi-Attention (300d LSTM)Time=≈6 min2018.03 | — | 0.403 | 48.7 | — | |
| Bi-Attention (300d MRU)Time=≈2 min2018.03 | — | 0.486 | 54.8 | — | |
| Bi-Attention (300d Sim. MRU)Time=≈25 sec2018.03 | — | 0.441 | 45.5 | — | |
| Bi-Attention (No Encoder)Time=≈17 sec2018.03 | — | 0.124 | 20.2 | — | |
| BIDAF2018.11 | — | — | 0.379 | 0.317 | |
| DECAPROP2018.11 | — | 0.645 | 71.9 | — | |
| DECAPROP2018.11 | — | — | 0.655 | 0.588 | |
| FH-RNN2018.11 | — | 0.496 | 56.7 | — | |
| TF-IDF max2018.03 | — | 0.13 | — | — | |
| TF-IDF max2018.11 | — | 0.13 | — | — | |
| TF-IDF Max2018.01 | — | 0.13 | — | — |