Flan
Benchmarks
TaskDatasetSOTA resultTrendResults
FLAN
0.211LDS Correlation
10
FLAN homogeneous federated setting (test)
84.3Average ROUGE-1
10
FLAN Heterogeneous Federated Setting
84.13Avg. ROUGE-1
8
FLAN subset Average (test)
56.7ROUGE-1
7
FLAN All (test)
57.5ROUGE-1
7
FLAN Dual (test)
56.3ROUGE-1
7
FLAN Single (test)
56.2ROUGE-1
7
FLAN 8-task subset: arc_challenge, cosmos_qa, definite_pronoun_resolution, glue_qqp, hellaswag, mnli, squad_v1, sst2
71Closed-book QA
7
FLAN 100K (test)
49.94Unigram F1
6
Flan
0.755Paraphrase
2