Language Understanding on MMLU (Five-client Non-IID, test)
48.5Law/Eth ScoreFedSSI
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| FedSSIBackbone=Qwen2.5-1.5B2026.05 | 48.5 | 74 | 53 | 48.1 | 70 | 58.7 | |
| FFA-LoRABackbone=Qwen2.5-1.5B2026.05 | 47.5 | 73 | 54.5 | 47.5 | 68 | 58.11 | |
| FedAvgBackbone=Qwen2.5-1.5B2026.05 | 47 | 72.5 | 53 | 48.6 | 69 | 58 | |
| FedProxBackbone=Qwen2.5-1.5B2026.05 | 47 | 73 | 52 | 49.2 | 69 | 58 | |
| FedALABackbone=Qwen2.5-1.5B2026.05 | 47 | 70.5 | 51.5 | 51.4 | 70 | 58.08 | |
| FedNLBackbone=Titans-Qwen2.5-1.5B2026.05 | 44.5 | 71.5 | 55 | 51.9 | 71.5 | 58.88 | |
| FedAvgBackbone=Llama-3.2-1B2026.05 | 34 | 45 | 38 | 37.7 | 51.5 | 41.2 | |
| FedSSIBackbone=Llama-3.2-1B2026.05 | 33.5 | 46 | 40 | 40.4 | 50.5 | 42.1 | |
| FedALABackbone=Llama-3.2-1B2026.05 | 33.5 | 42.5 | 38.5 | 40.4 | 48 | 40.59 | |
| FedNLBackbone=Titans-Llama-3.2-1B2026.05 | 33.5 | 43.5 | 38.5 | 43.7 | 54 | 42.64 | |
| FFA-LoRABackbone=Llama-3.2-1B2026.05 | 33 | 47 | 37 | 38.3 | 48.5 | 40.75 | |
| FedProxBackbone=Llama-3.2-1B2026.05 | 32.5 | 41 | 37.5 | 31.7 | 48.5 | 38.2 | |
| Fed-MambaBackbone=Mamba-1.4B2026.05 | 27 | 24 | 29.5 | 32.2 | 21 | 26.7 |