Safety Evaluation on BeaverTails & WildChat (test)
97.5Rule Adherence ScoreShadow-Level
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Shadow-LevelMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 97.5 | 95.96 | -0.8 | 5.39 | |
| Client-LevelMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 83.85 | 75.77 | -1.15 | 5.49 | |
| Step-LevelMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 83.65 | 72.12 | -1.34 | 5.13 | |
| KrumMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 72.88 | 60.96 | -1.68 | 5.64 | |
| FoolsGoldMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 72.88 | 57.88 | -1.82 | 5.27 | |
| TrimmedMeanMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 65.77 | 43.65 | -2.31 | 5.41 | |
| ResidualMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 63.65 | 41.15 | -2.34 | 5.23 | |
| FedAvgMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 61.35 | 37.69 | -2.49 | 5.47 |