Safety Evaluation on BeaverTails & LMSYS-Chat (test)
97.88Rule ScoreShadow-Level
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Shadow-LevelMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 97.88 | 96.92 | -0.87 | 5.17 | |
| KrumMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 72.88 | 59.04 | -2.36 | 5.16 | |
| FoolsGoldMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 72.12 | 60.96 | -2.24 | 5.16 | |
| Step-LevelMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 70.38 | 58.46 | -2.42 | 5.04 | |
| Client-LevelMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 69.04 | 56.73 | -2.52 | 5.14 | |
| FedAvgMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 60.77 | 44.23 | -3.07 | 5.15 | |
| ResidualMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 60.77 | 45 | -3.03 | 5.15 | |
| TrimmedMeanMalicious client ratio=30%, Backbone model=Qwen2.5-7B2026.01 | 60.38 | 43.46 | -3.06 | 4.75 |