Reward-hacking detection on School of Reward Hacks and Stanford Alpaca (test)
100F1 Scoreactivation-based monitoring system
Evaluation results
| Method | Links | |
|---|---|---|
| activation-based monitoring systemBackbone=Falcon3-7B, Adapter=control2026.03 | 100 | |
| activation-based monitoring systemBackbone=LLaMa 3.1-8B, Adapter=control2026.03 | 100 | |
| activation-based monitoring systemBackbone=Qwen2.5-Instruct 7B, Adapter=control2026.03 | 100 | |
| activation-based monitoring systemBackbone=LLaMa 3.1-8B, Adapter=hack2026.03 | 96.1 | |
| activation-based monitoring systemBackbone=LLaMa 3.1-8B, Adapter=mix502026.03 | 94.6 | |
| activation-based monitoring systemBackbone=LLaMa 3.1-8B, Adapter=mix902026.03 | 94.1 | |
| activation-based monitoring systemBackbone=Falcon3-7B, Adapter=mix052026.03 | 90.7 | |
| activation-based monitoring systemBackbone=Falcon3-7B, Adapter=mix502026.03 | 90.3 | |
| activation-based monitoring systemBackbone=Falcon3-7B, Adapter=hack2026.03 | 90.3 | |
| activation-based monitoring systemBackbone=Qwen2.5-Instruct 7B, Adapter=mix052026.03 | 89.7 | |
| activation-based monitoring systemBackbone=Falcon3-7B, Adapter=mix102026.03 | 86.8 | |
| activation-based monitoring systemBackbone=Falcon3-7B, Adapter=mix902026.03 | 86.8 | |
| activation-based monitoring systemBackbone=Qwen2.5-Instruct 7B, Adapter=mix502026.03 | 86.8 | |
| activation-based monitoring systemBackbone=Qwen2.5-Instruct 7B, Adapter=mix102026.03 | 86.2 | |
| activation-based monitoring systemBackbone=Qwen2.5-Instruct 7B, Adapter=mix902026.03 | 86.2 | |
| activation-based monitoring systemBackbone=LLaMa 3.1-8B, Adapter=mix102026.03 | 83.7 | |
| activation-based monitoring systemBackbone=Qwen2.5-Instruct 7B, Adapter=hack2026.03 | 78.4 | |
| activation-based monitoring systemBackbone=LLaMa 3.1-8B, Adapter=mix052026.03 | 76 |