Benchmarks

Task NameDataset NameSOTA ResultTrendResults

FLAN

0.211LDS Correlation

FLAN homogeneous federated setting (test)

84.3Average ROUGE-1

FLAN Heterogeneous Federated Setting

84.13Avg. ROUGE-1

FLAN subset Average (test)

56.7ROUGE-1

FLAN All (test)

57.5ROUGE-1

FLAN Dual (test)

56.3ROUGE-1

FLAN Single (test)

56.2ROUGE-1

FLAN 8-task subset: arc_challenge, cosmos_qa, definite_pronoun_resolution, glue_qqp, hellaswag, mnli, squad_v1, sst2

71Closed-book QA

FLAN 100K (test)

49.94Unigram F1

Flan

0.755Paraphrase