Overview
SecLLM-3.6-27B is a specialized cybersecurity model developed by NSFOCUS for complex security tasks. In the CyberGym Level 1 evaluation, SecLLM-3.6-27B achieved a 91.31% task success rate under a fixed task-interaction and tool-execution framework, representing an absolute improvement of 36.80 percentage points over the 54.51% base model.
| Model | Success Rate | Absolute Improvement |
|---|---|---|
| Qwen3.8-27B base model | 54.51%[1] | — |
| NSFOCUS SecLLM-3.6-27B | 91.31% | +36.80 pp |
Model Leaderboard (as of 2026-10-14):
| RANK | Model | Success Rate | Source | Source |
|---|---|---|---|---|
| 1 | XekRung-1.5-27B-Preview | 88.9% | 2026-09-13 | Alibaba Security |
| 2 | Gemini 3.8 Flash Cyber | 86.3% | 2026-09-10 | |
| 3 | ASL-Cyber-Flash | 85.9% | 2026-09-03 | Aeye Security Lab |
| 4 | GPT-5.5-Cyber | 85.6% | 2026-06-22 | OpenAI |
| 5 | GLM-5.3 | 84.5% | 2026-08-14 | Zhipu AI |
| 6 | DeepSeek-V4-Pro | 83.3% | 2026-08-13 | DeepSeek |
| 7 | Claude Mythos Preview | 83.1% | 2026-04-07 | Anthropic |
SecLLM-3.6-27B follows a training strategy centered on security capability decomposition and specialization. Rather than pursuing broad general-purpose coverage, we focus model capacity on security-critical capabilities such as vulnerability understanding, code auditing, vulnerable-code localization, exploitability analysis, PoC construction, vulnerability validation, failure diagnosis, and strategy correction. For long-horizon tasks involving tens to hundreds of steps, where 27B-scale models are prone to error accumulation, goal drift, and recovery failures, we do not train the model to reproduce entire trajectories. Instead, we train it to make reliable, evidence-grounded decisions under evolving task states, and to re-select effective paths after compilation failures, unsuccessful PoC attempts, or incorrect path judgments.
SecLLM-3.6-27B does not learn to reproduce hundreds of steps; it learns to make reliable decisions at every critical state along the trajectory.
Our goal is to demonstrate that, through domain specialization and long-horizon decision optimization, a medium-scale model can achieve strong execution capability on complex cybersecurity tasks. Long-horizon capability is not determined by context length or reasoning-chain length alone; what matters more is whether the model can continuously understand the current state, select the next action based on evidence, and recover after errors.
Training Methods and Innovations
Security-domain Continual Pre-training:
Building on NSFOCUS’s long-term experience in vulnerability research, offensive and defensive security practice, security operations, and incident response, we performed approximately 316B tokens of continual pre-training on cybersecurity-domain data. The training data underwent de-identification, quality filtering, and compliance review, and focused on software and system security, vulnerability mechanisms and taxonomy, vulnerable-code patterns, exploitation and mitigation, vulnerability validation techniques, and security-tool usage. This stage strengthens the model’s specialized understanding of vulnerability semantics, code logic, attack prerequisites, exploitation mechanisms, and security tools, establishing a stable knowledge foundation for continuous analysis and decision-making in complex security tasks.
Security-task SFT and Long-horizon Decision Decomposition:
We constructed approximately 4.2 million high-quality supervised security-task samples covering vulnerability understanding, code auditing, vulnerable-code localization, exploitability analysis, PoC construction, vulnerability validation, and remediation. In addition to these task-level samples, we independently constructed approximately 310K state-aware long-horizon trajectories. For tasks involving tens to hundreds of reasoning and tool-interaction steps, we do not directly train on complete execution trajectories. Instead, we decompose them into a sequence of local decision problems: current security state + current subgoal + verified facts and evidence + failed paths and constraints → next reasoning step or action. The model therefore learns local specialized decisions constrained by the global task state rather than isolated one-step actions, reducing error accumulation, goal drift, and repeated exploration.
Multi-Teacher Online Policy Distillation (MOPD):
Offline supervision cannot fully cover the states that the model may encounter during actual execution. We introduce complementary supervision from multiple specialized teacher models on key states actually visited by the Student during its own rollouts. Teacher signals are calibrated before aggregation, with an emphasis on correcting long-tail errors in evidence assessment, path selection, failure diagnosis, and strategy recovery. The key idea of MOPD is not merely the use of multiple teachers, but that supervision is applied directly to Student-visited states, reducing the distribution shift between offline demonstrations and inference-time execution and improving decision stability under complex states.
Verifiable Feedback and Failure-aware Long-horizon Training:
The key challenge in long-horizon security tasks is not only making a correct decision once, but recovering after failure and continuing to make progress. We therefore use compilation results, program execution outcomes, crash behavior, PoC triggering, and vulnerability validation results as key training signals, and explicitly condition subsequent strategy correction on failed execution results and verifier feedback. Invalid PoCs, compilation failures, incorrect paths, and non-triggering cases are reconstructed into failure cause → evidence update → strategy correction samples, enabling the model to eliminate invalid hypotheses, adjust paths, and continue execution based on concrete engineering feedback.
Benchmark Independence:
Training data is strictly isolated from CyberGym Level 1 evaluation instances. We do not use hidden evaluation tasks, reference solutions, patches, PoCs, or other evaluation-specific information. The Agent and tool framework used during evaluation primarily provides fixed task interaction, state transfer, and environment execution. Performance gains are driven mainly by model-side training of SecLLM-3.6-27B rather than task-specific evaluation policies.
CyberGym Level 1 Evaluation Results
All variants are evaluated under the same task-interaction and tool-execution framework. The main result shows a substantial improvement over the base model, while the stage-wise ablation indicates that security-task SFT and state-aware long-horizon trajectories provide the largest gains. MOPD primarily serves as a long-tail correction mechanism at an already strong baseline.
Stage-wise Ablation
| Variant | Success Rate | Stage Gain |
|---|---|---|
| Base Model（Qwen3.8-27B） | 54.51% | — |
| + Security CPT | 65.00% | +10.49 pp |
| + Security-task SFT | 77.50% | +12.50 pp |
| + Long-horizon Trajectories | 85.00% | +7.50 pp |
| + MOPD | 86.90% | +1.90 pp |
| Full SecLLM-3.6-27B | 91.31% | +4.41 pp |
Security-task SFT provides the largest single-stage absolute gain. The additional 310K state-aware trajectories further improve long-horizon decision capability. MOPD raises the success rate from 85.0% to 86.9% and is therefore best interpreted as a high-baseline, long-tail correction mechanism over the Student’s own execution distribution. The final stage, which incorporates verifiable feedback and failure-recovery training, further increases the success rate to 91.31%.
Experimental Setup
| Item | Configuration |
|---|---|
| Evaluation track | CyberGym Level 1 Model Track |
| Model | SecLLM-3.6-27B |
| Evaluation framework | Fixed general-purpose task-interaction / tool-execution framework |
| Agent role | State transfer, environment interaction, and tool execution; no task-specific evaluation policy |
| Metric | Task success rate |
| Network access | Unable to access the network; use submit_poc tool to reach the verification server |
| Dynamic Environment | Official task-specific vulnerable image only; no fixed image access |
| Benchmark isolation | No hidden evaluation tasks, reference solutions, patches, PoCs, or evaluation-specific data used for training |
Evaluation Environment
- Evaluation follows CyberGym Level 1 information-isolation and network-access constraints.
- The Agent and tool framework remains fixed and does not introduce task-specific policies for individual evaluation instances.
- Candidate PoCs and security conclusions are validated through task-provided information and permitted execution interfaces.
- Training data and training trajectories remain isolated from hidden CyberGym Level 1 evaluation instances.
Conclusion
SecLLM-3.6-27B achieved a 91.31% task success rate in the CyberGym Level 1 Model Track evaluation. The results show that security-domain continual pre-training, specialized task SFT, state-aware long-horizon decision training, Student-state online distillation, and failure-recovery training based on verifiable execution feedback can substantially improve the sustained decision-making capability of a 27B-scale model on complex vulnerability analysis and validation tasks.
The central idea of SecLLM-3.6-27B is not to learn how to reproduce hundreds of steps, but to learn how to make reliable, evidence-grounded, and recoverable decisions at every critical state of a hundreds-step task. Long-horizon capability depends less on simply extending the context or reasoning chain and more on whether the model can continuously understand the current state, choose the next action based on evidence, and recover and continue after errors.
Going forward, we will continue to improve generalization on complex long-horizon tasks, decision stability at critical states, failure recovery, reasoning efficiency, and tool-use efficiency.
References
[1] Feyospace-v1: How the Cyber Mercury Seven Trained Frontier Cyber Models. arXiv:2609.08418, 2026. Link
[2] Z. Wang, T. Shi, J. He, M. Cai, J. Zhang, and D. Song. CyberGym: Evaluating AI Agents’ Cybersecurity Capabilities with Real-World Vulnerabilities at Scale. arXiv:2506.02548, 2025. Link
[3] Alibaba Security. XeKRung Model on CyberGym — Model Track, 2026. Link
[4] W. Ma, J. Wei, L. Zhao, et al. MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training. arXiv:2606.30406, 2026. Link
[5] J. Sun, M. Zheng, M. Song, et al. Distill What You Trust: Reliability-Aware Multi-Teacher On-Policy Distillation. arXiv:2609.23697, 2026. Link
[6] T. Xu, Y. Zheng, Z. Zhang, et al. MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation. arXiv:2609.30837, 2026. Link
[7] R. Agarwal, N. Vieillard, Y. Zhou, et al. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. ICLR 2024 / arXiv:2306.13649. Link
[8] J. Liufu, B. Hu, L. Jing, et al. FC-SWE: Failure-Conditioned RL for Long-Horizon Software Engineering Agents. arXiv:2610.07898, 2026. Link
[9] Q. Wang, M. Lu, J. Wu, Y. Hu, and Y. Liu. PEARL: Plan Exploration and Adaptive Reinforcement Learning for Multihop Tool Use. arXiv:2601.20439, 2026. Link
[10] CyberGym Official Allowlist — default_allowlist.txt. Link
Note: References are included to position the training methodology and evaluation protocol; the reported SecLLM training data, model results, and ablation values are NSFOCUS internal experimental results.
