Cross-Task Generalization on Super-NaturalInstructions English Track (unseen clients)
62.2Weighted Avg Rouge-LFlexLoRA
Evaluation results
| Method | Links | |
|---|---|---|
| FlexLoRAFL Method=FedIT, Resource Distribution=Heavy-Tail-Strong2024.02 | 62.2 | |
| FlexLoRAFL Method=FedIT, Resource Distribution=Normal2024.02 | 62.01 | |
| FlexLoRAFL Method=FedIT, Resource Distribution=Heavy-Tail-Light2024.02 | 61.88 | |
| FlexLoRAFL Method=SLORA, Resource Distribution=Heavy-Tail-Strong2024.02 | 61.86 | |
| FlexLoRAFL Method=SLORA, Resource Distribution=Normal2024.02 | 61.67 | |
| HETLORAFL Method=FedIT, Resource Distribution=Heavy-Tail-Light2024.02 | 61.53 | |
| FlexLoRAFL Method=FedIT, Resource Distribution=Uniform2024.02 | 61.34 | |
| Homo RankFL Method=FedIT, Resource Distribution=Fixed (Homo Rank)2024.02 | 61.29 | |
| HETLORAFL Method=SLORA, Resource Distribution=Heavy-Tail-Strong2024.02 | 61.29 | |
| HETLORAFL Method=SLORA, Resource Distribution=Normal2024.02 | 61.14 | |
| HETLORAFL Method=FedIT, Resource Distribution=Heavy-Tail-Strong2024.02 | 61.06 | |
| HETLORAFL Method=FedIT, Resource Distribution=Normal2024.02 | 61.03 | |
| FlexLoRAFL Method=SLORA, Resource Distribution=Uniform2024.02 | 60.75 | |
| HETLORAFL Method=FedIT, Resource Distribution=Uniform2024.02 | 60.74 | |
| HETLORAFL Method=SLORA, Resource Distribution=Uniform2024.02 | 60.74 | |
| FlexLoRAFL Method=SLORA, Resource Distribution=Heavy-Tail-Light2024.02 | 60.4 | |
| Homo RankFL Method=SLORA, Resource Distribution=Fixed (Homo Rank)2024.02 | 60.01 | |
| HETLORAFL Method=SLORA, Resource Distribution=Heavy-Tail-Light2024.02 | 59.97 | |
| FlexLoRAFL Method=FedAvg, Resource Distribution=Uniform2024.02 | 58.07 | |
| FlexLoRAFL Method=FedAvg, Resource Distribution=Normal2024.02 | 57.78 | |
| FlexLoRAFL Method=FedAvg, Resource Distribution=Heavy-Tail-Strong2024.02 | 57.73 | |
| FlexLoRAFL Method=FedAvg, Resource Distribution=Heavy-Tail-Light2024.02 | 57.39 | |
| HETLORAFL Method=FedAvg, Resource Distribution=Uniform2024.02 | 56.85 | |
| Homo RankFL Method=FedAvg, Resource Distribution=Fixed (Homo Rank)2024.02 | 56.53 | |
| HETLORAFL Method=FedAvg, Resource Distribution=Normal2024.02 | 56.5 | |
| HETLORAFL Method=FedAvg, Resource Distribution=Heavy-Tail-Light2024.02 | 56.24 | |
| HETLORAFL Method=FedAvg, Resource Distribution=Heavy-Tail-Strong2024.02 | 55.74 |