MMLU Evaluation on Wizard
44.19AccuracyFedIT (ZERO-PAD)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FedIT (ZERO-PAD)Model=TinyLlama, Client Configuration=HETER2025.06 | 44.19 | 35,500 | |
| FLoRIST [τ=0.9]Model=TinyLlama, Client Configuration=HETER2025.06 | 41.51 | 361,600 | |
| FLoRIST [τ*]Model=TinyLlama, Client Configuration=HOMO2025.06 | 40.95 | 543,800 | |
| FLEXLORAModel=TinyLlama, Client Configuration=HOMO2025.06 | 39.75 | 142,000 | |
| FLoRIST [τ=0.9]Model=TinyLlama, Client Configuration=HOMO2025.06 | 38.92 | 630,900 | |
| FLoRIST [τ*]Model=TinyLlama, Client Configuration=HETER2025.06 | 38.2 | 136,000 | |
| FedITModel=TinyLlama, Client Configuration=HOMO2025.06 | 36.61 | 142,000 | |
| FLEXLORAModel=TinyLlama, Client Configuration=HETER2025.06 | 36.39 | 119,600 | |
| FFA-LoRAModel=TinyLlama, Client Configuration=HOMO2025.06 | 36.01 | 284,000 | |
| FLoRAModel=TinyLlama, Client Configuration=HOMO2025.06 | 34.2 | 17,800 | |
| FLoRAModel=TinyLlama, Client Configuration=HETER2025.06 | 33.74 | 5,000 | |
| FedITModel=LLAMA-3.2-1B, Client Configuration=HOMO2025.06 | 30.48 | 195,000 | |
| FLEXLORAModel=LLAMA-3.2-1B, Client Configuration=HOMO2025.06 | 30.03 | 195,000 | |
| FLoRIST [τ*]Model=LLAMA-3.2-1B, Client Configuration=HETER2025.06 | 29.79 | 478,200 | |
| FLoRIST [τ=0.9]Model=LLAMA-3.2-1B, Client Configuration=HOMO2025.06 | 29.62 | 511,900 | |
| FLoRIST [τ=0.9]Model=LLAMA-3.2-1B, Client Configuration=HETER2025.06 | 29.45 | 502,200 | |
| FLoRIST [τ*]Model=LLAMA-3.2-1B, Client Configuration=HOMO2025.06 | 29.31 | 511,900 | |
| FLEXLORAModel=LLAMA-3.2-1B, Client Configuration=HETER2025.06 | 29.13 | 164,400 | |
| FFA-LoRAModel=LLAMA-3.2-1B, Client Configuration=HETER2025.06 | 28.78 | 97,700 | |
| FLoRAModel=LLAMA-3.2-1B, Client Configuration=HOMO2025.06 | 28.57 | 24,400 | |
| FedIT (ZERO-PAD)Model=LLAMA-3.2-1B, Client Configuration=HETER2025.06 | 28.42 | 48,800 | |
| FFA-LoRAModel=LLAMA-3.2-1B, Client Configuration=HOMO2025.06 | 28.4 | 390,600 | |
| FLoRAModel=LLAMA-3.2-1B, Client Configuration=HETER2025.06 | 27.73 | 20,600 | |
| FFA-LoRAModel=TinyLlama, Client Configuration=HETER2025.06 | 23.75 | 71,000 |