Language Understanding on MMLU (test) - Model Specific Results
35.91Dolly MMLU ScoreLoRA
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| LoRAFoundation model=Llama, Strategy=Centralized, Fine-tuning algorithm=LoRA2024.09 | 35.91 | 29.18 | 31.68 | |
| FLORAFoundation model=Llama, Strategy=Homo, Fine-tuning algorithm=FLORA2024.09 | 30.99 | 29.85 | 34.26 | |
| FLORAFoundation model=TinyLlama, Strategy=Homo, Fine-tuning algorithm=FLORA2024.09 | 30.8 | 31.92 | 43.87 | |
| FedITFoundation model=Llama, Strategy=Homo, Fine-tuning algorithm=FedIT2024.09 | 29.67 | 29.41 | 33.43 | |
| FLORAFoundation model=Llama, Strategy=Heter, Fine-tuning algorithm=FLORA2024.09 | 28.5 | 29.54 | 27.91 | |
| LoRAFoundation model=TinyLlama, Strategy=Centralized, Fine-tuning algorithm=LoRA2024.09 | 27.99 | 28.03 | 29.13 | |
| Zero-paddingFoundation model=Llama, Strategy=Heter, Fine-tuning algorithm=Zero-padding2024.09 | 26.46 | 7.97 | 26.98 | |
| FLORAFoundation model=TinyLlama, Strategy=Heter, Fine-tuning algorithm=FLORA2024.09 | 18.45 | 29.69 | 41.48 | |
| FedITFoundation model=TinyLlama, Strategy=Homo, Fine-tuning algorithm=FedIT2024.09 | 16.35 | 30.02 | 42.51 | |
| Zero-paddingFoundation model=TinyLlama, Strategy=Heter, Fine-tuning algorithm=Zero-padding2024.09 | 15.76 | 29.56 | 40.79 |