List your company

ResearchFormerly WizWand

SCROLLS

Benchmarks

TaskDatasetSOTA resultTrendResults

Long-context language understanding suite

ZeroSCROLLS

33.5GovReport Score

24

Zero-shot long-context reasoning

ZeroSCROLLS

33.5Average Score

18

Long-context language understanding

SCROLLS (test)

47.4Average Score

18

Question Answering

Scrolls NarraQA

12.29Accuracy

10

Question Answering

Scrolls QAsper

14.8Accuracy

10

SCROLLS

14.83ROUGE-1

8

Long-context language understanding

SCROLLS (dev)

57.4GovRep ROUGE-1

7

Long-context Open-book Question Answering and Summarization

SCROLLS (val)

23.9NaQA F1 Score

6