Question Answering on TriviaQA (val)
74.8FT F1FLASH-Quad
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FLASH-QuadContext Length=4096, Model Size=~110M, Hardware=32 TPU-v4 cores, Head size (s)=5122022.02 | 74.8 | 1.67 | |
| Transformer+Context Length=4096, Model Size=~110M, Hardware=32 TPU-v4 cores2022.02 | 74.2 | 1 | |
| FLASHContext Length=4096, Model Size=~110M, Hardware=32 TPU-v4 cores, Head size (s)=512, Attention mechanism (first-to-all)=true2022.02 | 73.9 | 2.69 | |
| FLASHContext Length=4096, Model Size=~110M, Hardware=32 TPU-v4 cores, Head size (s)=5122022.02 | 73.3 | 2.6 | |
| FLASH-QuadContext Length=4096, Model Size=~110M, Hardware=32 TPU-v4 cores, Head size (s)=1282022.02 | 72.7 | 1.79 | |
| CombinerContext Length=4096, Model Size=~110M, Hardware=32 TPU-v4 cores2022.02 | 67.2 | 2.75 |