Robust Safety and Utility Evaluation in Federated Learning on BeaverTails & LMSYS-Chat
91.92Rule ScoreShadow-Level
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| Shadow-LevelBackbone=Llama3.1-8B, Malicious client ratio=30%, Base Framework=Safe-FedLLM2026.01 | 91.92 | 77.12 | -1.58 | 3.2 | |
| Step-LevelBackbone=Llama3.1-8B, Malicious client ratio=30%, Base Framework=Safe-FedLLM2026.01 | 88.85 | 76.35 | -1.73 | 3.07 | |
| Client-LevelBackbone=Llama3.1-8B, Malicious client ratio=30%, Base Framework=Safe-FedLLM2026.01 | 84.42 | 65.96 | -2.01 | 3.05 | |
| KrumBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 60.19 | 20.58 | -3.52 | 2.96 | |
| FoolsGoldBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 53.27 | 18.08 | -3.84 | 3.15 | |
| ResidualBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 53.08 | 12.12 | -4.01 | 3.14 | |
| FedAvgBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 51.73 | 14.81 | -3.97 | 3.18 | |
| TrimmedMeanBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 51.54 | 11.35 | -3.97 | 3.16 |