Long-context Open-book Question Answering and Summarization on SCROLLS (val)
23.9NaQA F1 ScoreMEGALODON
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| MEGALODONModel size=7B2024.04 | 23.9 | 28 | 13.1 | |
| LLAMA2-LModel size=7B, Context length=32K, Length extension pre-training tokens=500B2024.04 | 23.5 | 28.3 | 14.5 | |
| YaRNModel size=7B, Context length=128k2024.04 | 20.9 | 26.2 | 11.4 | |
| MPTModel size=7B, Context length=8K2024.04 | 18.8 | 24.7 | 8.8 | |
| LLAMA2Model size=7B, Context length=4K2024.04 | 18.8 | 19.8 | 10.1 | |
| XgenModel size=7B, Context length=8K2024.04 | 17.4 | 20.5 | 6.8 |