Reward Hacking on LeetCode with write access (test)
1Hack RateGen.-time suppression
Evaluation results
| Method | Links | |
|---|---|---|
| Gen.-time suppressionModel=Llama-3.2-3B, Write access=true2026.04 | 1 | |
| Adv. Mod. (additive)Model=Llama-3.2-3B, Alpha (α)=2.0, Write access=true2026.04 | 1.2 | |
| Adv. Mod. (multiplicative)Model=Llama-3.2-3B, Alpha (α)=2.0, Write access=true2026.04 | 1.8 | |
| No interventionModel=Llama-3.2-3B, Write access=true2026.04 | 1.9 | |
| No interventionModel=Phi-4-mini, Write access=true2026.04 | 2.5 | |
| Adv. Mod. (multiplicative)Model=Phi-4-mini, Alpha (α)=2.0, Write access=true2026.04 | 2.7 | |
| Adv. Mod. (additive)Model=Phi-4-mini, Alpha (α)=2.0, Write access=true2026.04 | 4.9 | |
| Gen.-time suppressionModel=Phi-4-mini, Write access=true2026.04 | 5.3 | |
| Adv. Mod. (multiplicative)Model=Llama-3.2-3B, alpha=2.02026.04 | 15.1 | |
| Adv. Mod. (multiplicative)Model=Phi-4-mini, alpha=2.02026.04 | 24.9 | |
| Adv. Mod. (additive)Model=Llama-3.2-3B, alpha=2.02026.04 | 47.8 | |
| Gen.-time suppressionModel=Llama-3.2-3B2026.04 | 53.4 | |
| Adv. Mod. (additive)Model=Phi-4-mini, alpha=2.02026.04 | 64.8 | |
| Gen.-time suppressionModel=Phi-4-mini2026.04 | 72.3 | |
| No interventionModel=Llama-3.2-3B2026.04 | 78.9 | |
| No interventionModel=Phi-4-mini2026.04 | 99.9 |