<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Detection Engineering on RockB</title><link>https://baeseokjae.github.io/tags/detection-engineering/</link><description>Recent content in Detection Engineering on RockB</description><image><title>RockB</title><url>https://baeseokjae.github.io/images/og-default.png</url><link>https://baeseokjae.github.io/images/og-default.png</link></image><generator>Hugo</generator><language>en-us</language><lastBuildDate>Tue, 11 Aug 2026 16:01:47 +0000</lastBuildDate><atom:link href="https://baeseokjae.github.io/tags/detection-engineering/index.xml" rel="self" type="application/rss+xml"/><item><title>CyberForge Review 2026: A Modular Cybersecurity LLM Agent Framework</title><link>https://baeseokjae.github.io/posts/cyberforge-llm-security-agents-2026/</link><pubDate>Tue, 11 Aug 2026 16:01:47 +0000</pubDate><guid>https://baeseokjae.github.io/posts/cyberforge-llm-security-agents-2026/</guid><description>Hands-on CyberForge review: a modular, provider-agnostic cybersecurity LLM agent framework for purple teaming, detection engineering, and lab-based adversary emulation.</description><content:encoded><![CDATA[<p>CyberForge is a Python-based, modular cybersecurity LLM agent framework built for lab environments, VM research, adversary emulation, detection engineering, and purple team automation. It decouples reusable <code>BaseAgent</code> and <code>BaseTask</code> classes, supports six or more LLM providers (Gemini, NVIDIA NIM, OpenRouter, OpenAI, Anthropic, and local Ollama), and ships a deterministic offline Lab Mock Mode so you can prototype agent workflows with zero API spend. This review walks through what it does, how it runs, and whether it earns a place in your SOC toolkit.</p>
<h2 id="what-is-cyberforge-a-modular-cybersecurity-llm-agent-framework">What Is CyberForge? A Modular Cybersecurity LLM Agent Framework</h2>
<p>CyberForge is an open-source project (created August 2026, Python) that treats security operations as a set of composable LLM-driven agents and tasks. Instead of a monolithic commercial platform, it gives you building blocks: a <code>BaseAgent</code> class that defines a role (SOC Analyst, Detection Engineer, Purple Team Specialist) and a <code>BaseTask</code> class that defines a unit of work. You wire them together through a centralized Scenario CLI.</p>
<p>The core idea is modularity. Rather than locking you into one vendor&rsquo;s model or one workflow, CyberForge lets you assemble the exact agent pipeline your team needs and run it against your own lab infrastructure. That design philosophy matters because the cybersecurity agentic AI market is exploding — projected to grow from USD 2.43 billion in 2026 to USD 9.63 billion by 2031, a 31.71% CAGR according to Mordor Intelligence. Teams are looking for flexible, low-cost ways to experiment before committing to expensive platforms.</p>
<h2 id="key-features-scenario-cli-multi-provider-llm-support-and-lab-mock-mode">Key Features: Scenario CLI, Multi-Provider LLM Support, and Lab Mock Mode</h2>
<p>CyberForge&rsquo;s most distinctive feature is its centralized Scenario CLI. You drive everything from a single entry point:</p>
<div class="highlight"><pre tabindex="0" style="color:#f8f8f2;background-color:#272822;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span>python main.py list
</span></span><span style="display:flex;"><span>python main.py run hello_agents
</span></span><span style="display:flex;"><span>python main.py run log_analysis
</span></span><span style="display:flex;"><span>python main.py run purple_team
</span></span></code></pre></div><p>The <code>list</code> command shows available scenarios, and <code>run</code> executes them. This is a deliberate contrast to sprawling, configuration-heavy security AI platforms — the whole framework is navigable from one command.</p>
<h3 id="multi-provider-llm-support">Multi-Provider LLM Support</h3>
<p>CyberForge is provider-agnostic. It supports Google Gemini, NVIDIA NIM, OpenCode/OpenRouter, OpenAI, Anthropic, and local Ollama. That breadth is a real advantage for SOC teams that want to avoid vendor lock-in, compare model quality on the same task, or run entirely on-premises with a self-hosted model.</p>
<h3 id="lab-mock-mode">Lab Mock Mode</h3>
<p>The standout feature for budget-conscious researchers is Lab Mock Mode. When no API key is set, CyberForge falls back to a deterministic, offline mock that simulates agent responses. This lets you validate your scenario wiring, test the CLI, and prototype workflows without spending a cent on cloud LLM calls. For a lab tool, that is a genuinely thoughtful touch — you can build and test the entire pipeline before you ever provision an API key.</p>
<h2 id="hands-on-running-hello_agents-log_analysis-and-purple_team-scenarios">Hands-On: Running hello_agents, log_analysis, and purple_team Scenarios</h2>
<p>The three flagship scenarios give a good sense of the framework&rsquo;s range.</p>
<p><strong>hello_agents</strong> is the smoke test. It spins up the configured agents, confirms the provider connector works, and verifies that tasks can be dispatched and completed. If you are testing a new provider or a fresh install, this is where you start.</p>
<p><strong>log_analysis</strong> is the detection-engineering workhorse. It points an agent at log data and asks it to identify suspicious activity, correlate events, and surface findings. This maps directly to the LLM-assisted defense category that is the largest research area in the field — 130 papers (17.2%) in the Awesome-LLM4Cybersecurity survey.</p>
<p><strong>purple_team</strong> is where CyberForge shines. It coordinates both offensive and defensive agents — an adversary-emulation agent generates attack behavior while a detection agent tries to catch it. This is the classic purple team loop, automated. It aligns with the fastest-growing research area: LLM-assisted attack, at 110 papers (14.6%) and climbing.</p>
<h2 id="architecture-deep-dive-baseagent-basetask-workflow-and-llm-provider-connector">Architecture Deep Dive: BaseAgent, BaseTask, Workflow, and LLM Provider Connector</h2>
<p>CyberForge&rsquo;s architecture is deliberately small and decoupled. The two primitives are:</p>
<ul>
<li><strong>BaseAgent</strong> — defines a role with a system prompt, a provider binding, and a set of capabilities. You subclass it to create a SOC Analyst, Detection Engineer, or Purple Team Specialist.</li>
<li><strong>BaseTask</strong> — defines a unit of work: an input, an expected output shape, and the agent that should handle it.</li>
</ul>
<p>A <strong>Workflow</strong> orchestrates tasks across agents, and an <strong>LLM Provider Connector</strong> abstracts the API differences between Gemini, OpenAI, Anthropic, NIM, and Ollama. Because the connector is a single interface, swapping providers is a configuration change rather than a code rewrite.</p>
<p>This decoupling is the framework&rsquo;s main architectural strength. It follows the same pattern that made agentic frameworks popular in other domains: separate the &ldquo;what&rdquo; (task) from the &ldquo;who&rdquo; (agent) from the &ldquo;how&rdquo; (provider). For security teams, that separation means you can reuse the same detection task against a cheap local model in development and a frontier model in production.</p>
<h2 id="lab-server-helpers-and-jupyter-notebook-vm-integration">Lab Server Helpers and Jupyter Notebook VM Integration</h2>
<p>CyberForge is built for lab environments, and it shows. It ships built-in lab server helpers — an HTTP server and a mock FTP server — so you can stand up realistic target infrastructure without standing up a full production environment. That is ideal for adversary emulation and detection testing where you need a controlled, disposable target.</p>
<p>It also integrates with Jupyter notebooks for VM research. You can drive agents from a notebook, inspect intermediate outputs, and iterate on prompts and tasks interactively. For researchers who live in notebooks, this lowers the barrier to entry considerably.</p>
<h2 id="cyberforge-vs-commercial-security-ai-platforms">CyberForge vs. Commercial Security AI Platforms</h2>
<table>
  <thead>
      <tr>
          <th>Dimension</th>
          <th>CyberForge</th>
          <th>Commercial Security AI Platforms</th>
      </tr>
  </thead>
  <tbody>
      <tr>
          <td>Cost</td>
          <td>Free, open source; Lab Mock Mode = $0</td>
          <td>Subscription/licensing, often per-seat or per-event</td>
      </tr>
      <tr>
          <td>Model choice</td>
          <td>6+ providers, swappable</td>
          <td>Usually locked to vendor&rsquo;s model</td>
      </tr>
      <tr>
          <td>Deployment</td>
          <td>Self-hosted, on-prem, air-gapped</td>
          <td>Often SaaS or vendor-managed</td>
      </tr>
      <tr>
          <td>Customization</td>
          <td>Full source access, subclass agents/tasks</td>
          <td>Limited to vendor APIs</td>
      </tr>
      <tr>
          <td>Support</td>
          <td>Community/self-service</td>
          <td>Vendor SLA and support</td>
      </tr>
      <tr>
          <td>Production readiness</td>
          <td>Lab-focused, DIY hardening</td>
          <td>Enterprise-grade, compliance-ready</td>
      </tr>
      <tr>
          <td>Best for</td>
          <td>Research, purple teaming, prototyping</td>
          <td>Large-scale production SOC operations</td>
      </tr>
  </tbody>
</table>
<p>The tradeoff is clear. Commercial platforms win on production hardening, compliance, and support. CyberForge wins on cost, flexibility, and the ability to run fully offline. For a lab or a small team that wants to learn and prototype, the open-source route is increasingly viable — a point reinforced by the growing ecosystem of open-source LLM security tooling.</p>
<h2 id="the-bigger-picture-agentic-ai-in-cybersecurity-market-stats--trends">The Bigger Picture: Agentic AI in Cybersecurity (Market Stats &amp; Trends)</h2>
<p>CyberForge is not an island — it sits at the center of a major industry shift. The numbers are striking:</p>
<ul>
<li>The cybersecurity agentic AI market is projected to grow from USD 2.43 billion (2026) to USD 9.63 billion by 2031, a 31.71% CAGR (Mordor Intelligence).</li>
<li>94% of respondents identified AI as the key driver of cybersecurity change for the year ahead (World Economic Forum).</li>
<li>77% of organizations already deploy AI for phishing detection, intrusion response, and SOC automation (WEF).</li>
<li>North America holds the largest market share; Asia-Pacific is the fastest-growing region at 32.71% CAGR.</li>
<li>SMEs are projected to grow at a 32.11% CAGR — directly favoring low-cost, open-source frameworks like CyberForge.</li>
</ul>
<p>The research landscape confirms the agentic shift. The Agent4Cybersecurity category is emerging in the Awesome-LLM4Cybersecurity survey, with 63 papers (8.3%) of 756+ total. LLM-assisted attack is the hottest growing area (110 papers, 14.6%), while LLM-assisted defense is the largest (130 papers, 17.2%). Frameworks like CyberForge are the practical embodiment of this research trend — turning papers into runnable agent pipelines.</p>
<h2 id="safety-responsible-use-and-the-lab-only-disclaimer">Safety, Responsible Use, and the Lab-Only Disclaimer</h2>
<p>Any security tool that automates adversary emulation carries responsibility. CyberForge is explicitly designed for lab environments, VM research, and authorized testing. The responsible-use framing is not optional — it is the difference between a research tool and a liability.</p>
<p>If you use CyberForge, follow these guardrails:</p>
<ul>
<li>Run it only against infrastructure you own or have explicit authorization to test.</li>
<li>Keep it in isolated VMs or containers, never against production systems.</li>
<li>Treat generated attack behavior as a research artifact, not a playbook for real engagements.</li>
<li>Review and validate all agent outputs before acting on them.</li>
<li>Understand your local laws and your organization&rsquo;s rules of engagement.</li>
</ul>
<p>The lab-only disclaimer is a feature, not a limitation. It keeps the framework safe to distribute and safe to learn with.</p>
<h2 id="who-should-use-cyberforge-and-who-shouldnt">Who Should Use CyberForge (and Who Shouldn&rsquo;t)</h2>
<p><strong>Use CyberForge if you are:</strong></p>
<ul>
<li>A security researcher or student learning purple teaming and detection engineering.</li>
<li>A SOC team prototyping agent workflows before committing to a commercial platform.</li>
<li>A small or mid-size organization (SME) that wants low-cost, self-hosted AI security tooling.</li>
<li>A detection engineer who wants to experiment with Sigma rule generation and log analysis agents.</li>
<li>Anyone who wants to test multiple LLM providers on the same security task.</li>
</ul>
<p><strong>Skip CyberForge if you are:</strong></p>
<ul>
<li>A large enterprise needing production-grade, compliance-ready, vendor-supported security AI.</li>
<li>A team that needs a fully managed SaaS solution with an SLA.</li>
<li>Someone looking for a turnkey product rather than a framework to assemble.</li>
</ul>
<h2 id="verdict-strengths-limitations-and-2026-outlook">Verdict: Strengths, Limitations, and 2026 Outlook</h2>
<p><strong>Strengths:</strong></p>
<ul>
<li>Genuinely modular architecture with clean <code>BaseAgent</code>/<code>BaseTask</code> separation.</li>
<li>Provider-agnostic with six or more LLM backends and a deterministic offline mock.</li>
<li>Zero-cost prototyping via Lab Mock Mode.</li>
<li>Built-in lab servers and Jupyter integration for realistic, disposable testing.</li>
<li>Aligns with the fastest-growing segment of the cybersecurity AI market.</li>
</ul>
<p><strong>Limitations:</strong></p>
<ul>
<li>Early-stage project (created August 2026, minimal community traction).</li>
<li>Lab-focused; you must harden it yourself for production.</li>
<li>No vendor support or SLA.</li>
<li>Documentation and examples are still maturing.</li>
</ul>
<p><strong>2026 Outlook:</strong> CyberForge arrives at exactly the right moment. With the agentic AI security market growing at 31.71% CAGR and SMEs driving demand for low-cost tooling, a modular, open-source, provider-agnostic framework fills a real gap. It is not a replacement for enterprise platforms, but for researchers, purple teams, and budget-conscious SOCs, it is a compelling way to get hands-on with LLM-driven security operations today.</p>
<h2 id="faq">FAQ</h2>
<p><strong>What is CyberForge?</strong>
CyberForge is an open-source, Python-based modular cybersecurity LLM agent framework for lab environments, VM research, adversary emulation, detection engineering, and purple team automation. It composes reusable agents and tasks through a centralized Scenario CLI.</p>
<p><strong>Which LLM providers does CyberForge support?</strong>
It supports Google Gemini, NVIDIA NIM, OpenCode/OpenRouter, OpenAI, Anthropic, and local Ollama — six or more backends — plus a deterministic offline Lab Mock Mode that runs without any API key.</p>
<p><strong>What is Lab Mock Mode in CyberForge?</strong>
Lab Mock Mode is a deterministic, offline fallback that simulates agent responses when no API key is set. It lets you prototype and test scenario workflows with zero cloud LLM spend.</p>
<p><strong>Is CyberForge safe to use?</strong>
Yes, when used responsibly. It is explicitly designed for lab environments and authorized testing only. You should run it only against infrastructure you own or are authorized to test, and always validate agent outputs.</p>
<p><strong>How does CyberForge compare to commercial security AI platforms?</strong>
CyberForge is free, open source, self-hosted, and provider-agnostic, making it ideal for research and prototyping. Commercial platforms offer production hardening, compliance, and vendor support, but at higher cost and with less flexibility.</p>
]]></content:encoded></item></channel></rss>