Federated Domain-Specific Fine-tuning on Code, Math, Medical, and Finance Benchmarks
61.19Average In-Domain PerformanceFedSRD
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| FedSRDBase Model=Qwen2-7B, Comm. Cost (MB)=612025.10 | 61.19 | 68.29 | 36.19 | 77.77 | 78.94 | 64.55 | 48.48 | 69.93 | 45.36 | |
| FedSRD-eBase Model=Qwen2-7B, Comm. Cost (MB)=612025.10 | 61.13 | 68.09 | 36.56 | 77.8 | 78.1 | 64.21 | 48.9 | 70.2 | 45.2 | |
| FedProxBase Model=Qwen2-7B, Comm. Cost (MB)=6162025.10 | 58.27 | 63.42 | 36.58 | 62.13 | 77.34 | 65.77 | 49.96 | 66.26 | 44.72 | |
| FedICUBase Model=Qwen2-7B, Comm. Cost (MB)=4722025.10 | 57.98 | 60.37 | 38.91 | 60.33 | 78.72 | 65.86 | 48.83 | 66.19 | 44.62 | |
| FedAvgBase Model=Qwen2-7B, Comm. Cost (MB)=6162025.10 | 57.04 | 59.15 | 37.35 | 61.67 | 75.23 | 65.16 | 49.04 | 64.29 | 44.46 | |
| Fed-DAREBase Model=Qwen2-7B, Comm. Cost (MB)=1582025.10 | 57.01 | 60.37 | 31.91 | 61.12 | 75.3 | 66.03 | 50.84 | 65.96 | 44.58 | |
| FlexLoRABase Model=Qwen2-7B, Comm. Cost (MB)=6162025.10 | 55.89 | 63.41 | 35.02 | 58.83 | 68.3 | 67.68 | 50.53 | 58.61 | 44.7 | |
| FFA-LoRABase Model=Qwen2-7B, Comm. Cost (MB)=3082025.10 | 53.36 | 68.29 | 17.9 | 51.55 | 57.02 | 67.76 | 48.92 | 69.37 | 46.08 | |
| FedSRDBase Model=Llama3.2-3B, Comm. Cost (MB)=742025.10 | 35.12 | 31.1 | 49.42 | 41.95 | 41.62 | 43.44 | 39.13 | 28.96 | 5.36 | |
| FedSRD-eBase Model=Llama3.2-3B, Comm. Cost (MB)=742025.10 | 34.26 | 29.27 | 45.87 | 41.63 | 41.17 | 42.22 | 39.47 | 30.48 | 5.88 | |
| FedICUBase Model=Llama3.2-3B, Comm. Cost (MB)=5682025.10 | 33.91 | 31.1 | 49.42 | 40.15 | 41.07 | 40.83 | 37.56 | 25.09 | 5.26 | |
| FedAvgBase Model=Llama3.2-3B, Comm. Cost (MB)=7422025.10 | 33.9 | 26.85 | 47.86 | 40.45 | 41.53 | 41.96 | 38.08 | 28.96 | 5.56 | |
| Fed-DAREBase Model=Llama3.2-3B, Comm. Cost (MB)=1902025.10 | 33.57 | 26.6 | 48.25 | 39.28 | 40.04 | 40.4 | 38.04 | 26.05 | 5.26 | |
| FedProxBase Model=Llama3.2-3B, Comm. Cost (MB)=7422025.10 | 33.43 | 28.05 | 48.25 | 35.98 | 40.06 | 42.75 | 37.82 | 28.05 | 5.7 | |
| FlexLoRABase Model=Llama3.2-3B, Comm. Cost (MB)=7422025.10 | 32.36 | 28.75 | 47.47 | 36.29 | 41.24 | 42.22 | 37.09 | 27.37 | 5.68 | |
| FFA-LoRABase Model=Llama3.2-3B, Comm. Cost (MB)=3712025.10 | 28.39 | 3.05 | 46.3 | 39.95 | 40.11 | 40.23 | 36.02 | 24.79 | 5.1 |