# Pen-Strategist: A Reasoning Framework for Penetration Testing Strategy Formation and Analysis Source: [Pen-Strategist: A Reasoning Framework for Penetration Testing Strategy Formation and Analysis]() Collection index: [llms.txt]() Data updated: 2026-05-08T05:00:04.262Z Authors: Yasod Ginige, Pasindu Marasinghe, Sajal Jain, Suranga Seneviratne Published: 2026-05-06T05:02:40.000Z ## Abstract Cyber threats are rapidly increasing, expanding their impact from large-scale enterprises to government services and individual users, making robust security systems increasingly essential. However, a significant shortage of skilled cybersecurity professionals exacerbates this challenge. While recent research has explored automating tasks such as penetration testing using LLM-based agents, existing frameworks often perform poorly due to limited capability in strategy formulation, domain-specific reasoning, and accurate action and tool selection. To overcome these limitations, we propose Pen-Strategist framework, consisting of a novel domain-specific reasoning model that derives pentesting strategies via logical reasoning and a classifier that converts the strategies into actionable steps. First, we construct a reasoning dataset containing logical explanations for both strategy derivation and step selection in pentesting scenarios. We then fine-tune a Qwen-3-14B model for strategy generation using reinforcement learning. Evaluation on the test split of the dataset demonstrates a 87% improvement in strategy derivation performance compared to the baseline. Furthermore, we integrate the fine-tuned Pen-Strategist model into existing automated pentesting frameworks, such as PentestGPT, and evaluate its performance on vulnerable machines, achieving a 47.5% improvement in subtask completion while surpassing the baseline GPT-5. Further experiments on the CTFKnow benchmark show an 18% performance gain over the base model. For step prediction, we train a semantic-based CNN classifier, which outperforms commercial LLMs by 28% and enhances execution stability. Finally, we conduct a user study to qualitatively assess the generated strategies, and Pen-Strategist demonstrates superior performance compared to the Claude-4.6-Sonnet. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Pentesting Strategy Generation on Pentesting Scenarios (test)](): Pentesting Strategy Generation; Pentesting Scenarios (test); Strategy Success Rate: 73. - [MCP Server Prediction on Pen-Strategist (test)](): MCP Server Prediction; Pen-Strategist (test); Accuracy: 48.88. - [Step Prediction on Pen-Strategist (test)](): Step Prediction; Pen-Strategist (test); Accuracy: 82.87. - [Pentesting on PicoCTF](): Pentesting; PicoCTF; Success Rate (out of 5): 60. - [Pentesting Explanation Generation on Pentesting Scenarios (test)](): Pentesting Explanation Generation; Pentesting Scenarios (test); Explanation Score: 71. - [Capture The Flag (CTF) on CTF Known](): Capture The Flag (CTF); CTF Known; Web Score: 81.31. ## Original sources and implementation [Original paper]() [PDF]()