Safety and Utility Evaluation on BeaverTails & WildChat
97.5Rule AdherenceShadow-Level
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Shadow-LevelBackbone=Qwen2.5-7B, Malicious client ratio=30%2026.01 | 97.5 | 95.96 | -0.8 | 5.39 | |
| Client-LevelBackbone=Qwen2.5-7B, Malicious client ratio=30%2026.01 | 83.85 | 75.77 | -1.15 | 5.49 | |
| Step-LevelBackbone=Qwen2.5-7B, Malicious client ratio=30%2026.01 | 83.65 | 72.12 | -1.34 | 5.13 | |
| Client-LevelBackbone=Llama3.1-8B, Malicious client ratio=30%, Base Framework=Safe-FedLLM2026.01 | 82.12 | 54.81 | -1.83 | 3.95 | |
| Step-LevelBackbone=Llama3.1-8B, Malicious client ratio=30%, Base Framework=Safe-FedLLM2026.01 | 80.58 | 52.5 | -1.89 | 3.02 | |
| Shadow-LevelBackbone=Llama3.1-8B, Malicious client ratio=30%, Base Framework=Safe-FedLLM2026.01 | 79.42 | 51.35 | -1.86 | 4.1 | |
| FoolsGoldBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 51.54 | 7.69 | -3.18 | 3.91 | |
| ResidualBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 50.58 | 7.31 | -3.32 | 4.14 | |
| FedAvgBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 49.42 | 7.88 | -3.45 | 3.95 | |
| TrimmedMeanBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 47.51 | 5.96 | -3.31 | 3.91 | |
| KrumBackbone=Llama3.1-8B, Malicious client ratio=30%2026.01 | 45.96 | 4.42 | -3.37 | 4.3 |