Natural Instructions
Benchmarks
TaskDatasetSOTA resultTrendResults
Natural Instructions (test)
97.9Rouge-L
90
Super-NaturalInstructions English Track (unseen clients)
62.2Weighted Avg Rouge-L
27
Natural Instructions Meta Non-IID
34.81Rouge-L
22
Natural Instructions (NI)
48.8Speedup
10
Natural Instructions (val)
2.391Answer Verification
7
Natural Instructions 700 domains (held-out prompts)
62.9Top-1 Accuracy
5
Natural Instructions task459_matres_static_classification
69Correctness
3
Natural Instructions task457_matres_conditional_classification
87Correctness
3
Natural Instructions task108_contextualabusedetection_classification
75Correctness
3
Natural Instructions task022_cosmosqa_passage_inappropriate_binary
80Correctness
3
Natural Instructions task021_mctaco_grammatical_logical
0.5Correctness
3
Natural Instructions (test)
3.076Answerability Classification
3