Score Inflation Attack on Chatbot Benchmarks AlpacaEval ArenaHard
2.01Qwen ScoreBITE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| BITECategory=Ours2026.05 | 2.01 | 1.909 | 1.347 | 1.731 | 1.356 | |
| Fake CompletionCategory=Prompt Injection2026.05 | 1.287 | 1.214 | 1.08 | 1.089 | 1.103 | |
| PAIRCategory=Jailbreak2026.05 | 1.284 | 1.856 | 1.233 | 1.337 | 0.869 | |
| Escape ChrCategory=Prompt Injection2026.05 | 1.274 | 0.858 | 0.96 | 1.186 | 1.083 | |
| AutoDANCategory=Jailbreak2026.05 | 0.941 | 1.365 | 1.166 | 1.489 | 1.091 | |
| Null ModelCategory=Prompt Injection2026.05 | 0.878 | 0.423 | 0.741 | 1.092 | 1.056 | |
| NaiveCategory=Prompt Injection2026.05 | 0.833 | 1.016 | 0.763 | 0.862 | 0.65 | |
| Context IgnoreCategory=Prompt Injection2026.05 | 0.65 | 0.713 | 0.764 | 0.725 | 0.727 | |
| TAPCategory=Jailbreak2026.05 | 0.622 | 0.871 | 0.656 | 0.693 | 0.519 |