# Cyber-Zero: Training Cybersecurity Agents without Runtime

Source: [Cyber-Zero: Training Cybersecurity Agents without Runtime](<https://www.sota2.com/research/paper/68fa7ae51e209131cea85c05>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:57:20.885Z

Authors: Terry Yue Zhuo, Dingmin Wang, Hantian Ding, Varun Kumar, Zijian Wang

Published: 2025-07-29T18:10:37.000Z

## Abstract

Large Language Models (LLMs) have achieved remarkable success in software engineering tasks when trained with executable runtime environments, particularly in resolving GitHub issues. However, such runtime environments are often unavailable in other domains, especially cybersecurity, where challenge configurations and execution contexts are ephemeral or restricted. We present Cyber-Zero, the first runtime-free framework for synthesizing high-quality agent trajectories to train cybersecurity LLMs. Cyber-Zero leverages publicly available CTF writeups and employs persona-driven LLM simulation to reverse-engineer runtime behaviors and generate realistic, long-horizon interaction sequences without actual environments. Using trajectories synthesized by Cyber-Zero, we train LLM-based agents that achieve up to 13.1% absolute performance gains over baseline models on three prominent CTF benchmarks: InterCode-CTF, NYU CTF Bench, and Cybench. Our best model, Cyber-Zero-32B, establishes new state-of-the-art performance among open-weight models, matching the capabilities of proprietary systems like DeepSeek-V3-0324 and Claude-3.5-Sonnet while offering superior cost-effectiveness, and demonstrating that runtime-free trajectory synthesis can effectively democratize the development of state-of-the-art cybersecurity agents.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Cyber-Threat Intelligence Prediction on LLM-CTI (Prediction)](<https://www.sota2.com/research/sota/cyber-threat-intelligence-prediction-on-llm-cti-prediction.md>): Cyber-Threat Intelligence Prediction; LLM-CTI (Prediction); Exploit Likelihood (AUC): 0.703.

- [Cyber-Threat Intelligence Mitigation on LLM-CTI Mitigation](<https://www.sota2.com/research/sota/cyber-threat-intelligence-mitigation-on-llm-cti-mitigation.md>): Cyber-Threat Intelligence Mitigation; LLM-CTI Mitigation; Patch Recommendation F1: 56.7.

- [Cyber-Threat Intelligence Attribution on LLM-CTI Attribution](<https://www.sota2.com/research/sota/cyber-threat-intelligence-attribution-on-llm-cti-attribution.md>): Cyber-Threat Intelligence Attribution; LLM-CTI Attribution; Threat Actor Linking Accuracy: 52.6.

- [Cyber-Threat Intelligence Contextualization on LLM-CTI Contextualization](<https://www.sota2.com/research/sota/cyber-threat-intelligence-contextualization-on-llm-cti-contextualization.md>): Cyber-Threat Intelligence Contextualization; LLM-CTI Contextualization; Affected Systems F1: 41.8.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2508.00910>)

[PDF](<https://arxiv.org/pdf/2508.00910.pdf>)
