# Context-Based Adversarial Attacks on AI Code Generators: Vulnerability Analysis and Implications

Source: [Context-Based Adversarial Attacks on AI Code Generators: Vulnerability Analysis and Implications](<https://www.sota2.com/research/paper/6a28f38290966ade919c1a74>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-11T05:00:14.404Z

Authors: Walther A. Del Orbe, John D. Hastings, Varghese Vaidyan

Published: 2026-06-09T14:51:59.000Z

## Abstract

AI-powered code generation systems have transformed software development but introduce critical inference-time security vulnerabilities. This research presents a systematic investigation of context-based adversarial attacks, where strategically crafted contextual inputs, including comments, documentation, variable names, bias large language models toward generating exploitable code. Through 2,800 controlled experiments across CodeT5+, CodeLlama, GPT-3.5-Turbo, and GPT-4, we quantify attack effectiveness and defense mechanisms. Results demonstrate that adversarial conditions increase vulnerability generation 10.7x (from 3.5% to 37.4%), with direct instruction attacks achieving 100% success on GPT-3.5-Turbo. Cross-model transferability reaches 60-100%, indicating systemic architectural vulnerabilities rather than model-specific flaws. Our dual-layer defense framework achieves 89.1% detection rate with 0.3% false positives and 520ms latency, demonstrating practical feasibility for real-time deployment in development environments.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Adversarial Attack Detection on Held-out n = 560 (20% stratified test)](<https://www.sota2.com/research/sota/adversarial-attack-detection-on-held-out-n-560-20-percent-stratified-test.md>): Adversarial Attack Detection; Held-out n = 560 (20% stratified test); Detection Rate: 89.1.

- [Vulnerability Generation on Vulnerability Generation Prompt Dataset (test)](<https://www.sota2.com/research/sota/vulnerability-generation-on-vulnerability-generation-prompt-dataset-test.md>): Vulnerability Generation; Vulnerability Generation Prompt Dataset (test); VGR: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2606.10945>)

[PDF](<https://arxiv.org/pdf/2606.10945.pdf>)
