Federated Instruction Tuning (SFT) on Alpaca GPT4 (held-out)
0.8979LossFedACT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| FedACTCommunication Rounds=200, Number of Clients=20, Participation Rate=10%, Batch Size=16, K (Local Iterations)=102026.07 | 0.8979 | 2.4544 | 73.47 | 0.8613 | |
| FedAvgCommunication Rounds=200, Number of Clients=20, Participation Rate=10%, Batch Size=16, K (Local Iterations)=102026.07 | 0.8997 | 2.4589 | 73.42 | 0.865 | |
| FedProxCommunication Rounds=200, Number of Clients=20, Participation Rate=10%, Batch Size=16, K (Local Iterations)=102026.07 | 0.9002 | 2.46 | 73.41 | 0.8653 | |
| SCAFFOLDCommunication Rounds=200, Number of Clients=20, Participation Rate=10%, Batch Size=16, K (Local Iterations)=102026.07 | 0.9007 | 2.4614 | 73.4 | 0.8674 | |
| FedAdamCommunication Rounds=200, Number of Clients=20, Participation Rate=10%, Batch Size=16, K (Local Iterations)=102026.07 | 0.9046 | 2.4708 | 73.31 | 0.8749 | |
| FedAdamWCommunication Rounds=200, Number of Clients=20, Participation Rate=10%, Batch Size=16, K (Local Iterations)=102026.07 | 0.9047 | 2.4712 | 73.31 | 0.8735 | |
| FedLADACommunication Rounds=200, Number of Clients=20, Participation Rate=10%, Batch Size=16, K (Local Iterations)=102026.07 | 0.9112 | 2.4873 | 73.16 | 0.885 | |
| BaseCommunication Rounds=200, Number of Clients=20, Participation Rate=10%, Batch Size=16, K (Local Iterations)=102026.07 | 1.0682 | 2.9102 | 70.32 | 1.3236 |