Out-of-Domain Generalization on AGIEval Law (test)
43.41Average OOD AccuracyFFA-LoRA
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| FFA-LoRABase Model=Qwen2-7B, Comm. Cost (MB)=3082025.10 | 43.41 | 46.92 | 39.89 | |
| FedSRD-eBase Model=Qwen2-7B, Comm. Cost (MB)=612025.10 | 42.59 | 45.82 | 39.35 | |
| FedSRDBase Model=Qwen2-7B, Comm. Cost (MB)=612025.10 | 42.58 | 45.93 | 39.22 | |
| FedICUBase Model=Qwen2-7B, Comm. Cost (MB)=4722025.10 | 42.56 | 48.44 | 36.28 | |
| Fed-DAREBase Model=Qwen2-7B, Comm. Cost (MB)=1582025.10 | 41.72 | 48.37 | 35.07 | |
| FedAvgBase Model=Qwen2-7B, Comm. Cost (MB)=6162025.10 | 41.02 | 48.25 | 33.79 | |
| FedProxBase Model=Qwen2-7B, Comm. Cost (MB)=6162025.10 | 40.72 | 48.62 | 32.82 | |
| FlexLoRABase Model=Qwen2-7B, Comm. Cost (MB)=6162025.10 | 40.49 | 47.81 | 33.17 | |
| FedSRD-eBase Model=Llama3.2-3B, Comm. Cost (MB)=742025.10 | 27.95 | 28.81 | 27.08 | |
| FedSRDBase Model=Llama3.2-3B, Comm. Cost (MB)=742025.10 | 26.17 | 25.54 | 26.79 | |
| FFA-LoRABase Model=Llama3.2-3B, Comm. Cost (MB)=3712025.10 | 26.12 | 24.63 | 27.6 | |
| FlexLoRABase Model=Llama3.2-3B, Comm. Cost (MB)=7422025.10 | 26.08 | 24.92 | 27.24 | |
| FedICUBase Model=Llama3.2-3B, Comm. Cost (MB)=5682025.10 | 25.76 | 24.77 | 26.75 | |
| FedAvgBase Model=Llama3.2-3B, Comm. Cost (MB)=7422025.10 | 24.36 | 22.47 | 26.24 | |
| Fed-DAREBase Model=Llama3.2-3B, Comm. Cost (MB)=1902025.10 | 24.25 | 21.94 | 26.56 | |
| FedProxBase Model=Llama3.2-3B, Comm. Cost (MB)=7422025.10 | 23.82 | 21.58 | 26.06 |