# Reward Hacking on LeetCode with write access (test) Source: [Reward Hacking on LeetCode with write access (test)]() Collection index: [llms.txt]() Data updated: 2026-04-03T07:34:10.923Z Evaluation of model performance on the test split of the LeetCode environment (with write access context), specifically assessing the model's propensity for Reward Hacking, measured by the Hack Rate. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Reward Hacking | | Dataset (including any stated split) | LeetCode with write access (test) | | Metric | Hack Rate | | Unit | PERCENT | | Direction | Lower is better | | Results loaded | 16 of 16 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Hack Rate](): Lower is better; unit: PERCENT. ## Recorded results | Method | Hack Rate | Result date | Report paper | | --- | --- | --- | --- | | Gen.-time suppression | 1 | Not recorded | 69cf49700636e5bf4afcd920 | | Adv. Mod. (additive) | 1.2 | Not recorded | 69cf49700636e5bf4afcd920 | | Adv. Mod. (multiplicative) | 1.8 | Not recorded | 69cf49700636e5bf4afcd920 | | No intervention | 1.9 | Not recorded | 69cf49700636e5bf4afcd920 | | No intervention | 2.5 | Not recorded | 69cf49700636e5bf4afcd920 | | Adv. Mod. (multiplicative) | 2.7 | Not recorded | 69cf49700636e5bf4afcd920 | | Adv. Mod. (additive) | 4.9 | Not recorded | 69cf49700636e5bf4afcd920 | | Gen.-time suppression | 5.3 | Not recorded | 69cf49700636e5bf4afcd920 | | Adv. Mod. (multiplicative) | 15.1 | Not recorded | 69cf49700636e5bf4afcd920 | | Adv. Mod. (multiplicative) | 24.9 | Not recorded | 69cf49700636e5bf4afcd920 | | Adv. Mod. (additive) | 47.8 | Not recorded | 69cf49700636e5bf4afcd920 | | Gen.-time suppression | 53.4 | Not recorded | 69cf49700636e5bf4afcd920 | | Adv. Mod. (additive) | 64.8 | Not recorded | 69cf49700636e5bf4afcd920 | | Gen.-time suppression | 72.3 | Not recorded | 69cf49700636e5bf4afcd920 | | No intervention | 78.9 | Not recorded | 69cf49700636e5bf4afcd920 | | No intervention | 99.9 | Not recorded | 69cf49700636e5bf4afcd920 | ### Gen.-time suppression Hack Rate: 1. Result ID: 69cf6c4d47f34ce924b8ea34. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B","Write access":"true"}. This record does not assert independent reproduction. ### Adv. Mod. (additive) Hack Rate: 1.2. Result ID: 69cf6c4d47f34ce924b8ea35. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B","Alpha (α)":"2.0","Write access":"true"}. This record does not assert independent reproduction. ### Adv. Mod. (multiplicative) Hack Rate: 1.8. Result ID: 69cf6c4d47f34ce924b8ea36. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B","Alpha (α)":"2.0","Write access":"true"}. This record does not assert independent reproduction. ### No intervention Hack Rate: 1.9. Result ID: 69cf6c4d47f34ce924b8ea33. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B","Write access":"true"}. This record does not assert independent reproduction. ### No intervention Hack Rate: 2.5. Result ID: 69cf6c4d47f34ce924b8ea2f. [Reporting paper]() Recorded evaluation configuration: {"Model":"Phi-4-mini","Write access":"true"}. This record does not assert independent reproduction. ### Adv. Mod. (multiplicative) Hack Rate: 2.7. Result ID: 69cf6c4d47f34ce924b8ea32. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Phi-4-mini","Alpha (α)":"2.0","Write access":"true"}. This record does not assert independent reproduction. ### Adv. Mod. (additive) Hack Rate: 4.9. Result ID: 69cf6c4d47f34ce924b8ea31. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Phi-4-mini","Alpha (α)":"2.0","Write access":"true"}. This record does not assert independent reproduction. ### Gen.-time suppression Hack Rate: 5.3. Result ID: 69cf6c4d47f34ce924b8ea30. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Phi-4-mini","Write access":"true"}. This record does not assert independent reproduction. ### Adv. Mod. (multiplicative) Hack Rate: 15.1. Result ID: 69cf6c4d47f34ce924b8ea46. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B","alpha":"2.0"}. This record does not assert independent reproduction. ### Adv. Mod. (multiplicative) Hack Rate: 24.9. Result ID: 69cf6c4d47f34ce924b8ea42. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Phi-4-mini","alpha":"2.0"}. This record does not assert independent reproduction. ### Adv. Mod. (additive) Hack Rate: 47.8. Result ID: 69cf6c4d47f34ce924b8ea45. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B","alpha":"2.0"}. This record does not assert independent reproduction. ### Gen.-time suppression Hack Rate: 53.4. Result ID: 69cf6c4d47f34ce924b8ea44. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B"}. This record does not assert independent reproduction. ### Adv. Mod. (additive) Hack Rate: 64.8. Result ID: 69cf6c4d47f34ce924b8ea41. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Phi-4-mini","alpha":"2.0"}. This record does not assert independent reproduction. ### Gen.-time suppression Hack Rate: 72.3. Result ID: 69cf6c4d47f34ce924b8ea40. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Model":"Phi-4-mini"}. This record does not assert independent reproduction. ### No intervention Hack Rate: 78.9. Result ID: 69cf6c4d47f34ce924b8ea43. [Reporting paper]() Recorded evaluation configuration: {"Model":"Llama-3.2-3B"}. This record does not assert independent reproduction. ### No intervention Hack Rate: 99.9. Result ID: 69cf6c4d47f34ce924b8ea3f. [Reporting paper]() Recorded evaluation configuration: {"Model":"Phi-4-mini"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.