Back to all articles

Evaluating AI in the SOC: Operational Metrics and Governance from RSAC 2026

Enterprise security leaders from the financial and manufacturing sectors shared results from a six-month trial integrating AI into their Security Operations Centers. The findings demonstrate that while large language models effectively reduce analyst fatigue and time-to-discovery, safe deployment requires strict read-only architectures and human-in-the-loop governance.

Triage Security Media Team
3 min read

At the RSAC 2026 Conference in San Francisco, enterprise security leaders addressed the operational pressure to integrate artificial intelligence into security operations centers (SOCs). To determine where AI provides measurable value and where it introduces unmanaged risk, Shilpi Mittal, overseeing a Fortune 500 food manufacturing company, and Ankit Gupta, protecting a financial institution, conducted structured six-month trials. They shared their findings during the session, "We Put AI in Our SOC — Here’s What Worked and What Didn't."

Both environments require rigorous safeguarding against security incidents, though their specific operational constraints differ. To test the technology responsibly, Mittal and Gupta established defined pilot programs to measure performance, fatigue reduction, and system reliability.

Deploying AI in a manufacturing environment

Mittal’s team deployed a large language model (LLM) as a read-only triage assistant within their case workflow. The tool evaluated telemetry from endpoint detection and response (EDR), cloud systems, applications, and operational technology (OT) monitoring feeds.

The pilot yielded quantifiable improvements in standard operational metrics. Based on initial processing, mean time to discovery (MTD) improved by 26% to 36%, mean time to response (MTTR) improved by 22%, and false positives dropped by 16 points.

Because operational downtime in manufacturing directly impacts production lines and worker safety, the team established strict system boundaries. AI was prohibited from directly interacting with programmable logic controllers (PLCs), SCADA systems, or any production equipment. The security team also enforced human approval gates, strict tool allow lists, mandatory citations for AI outputs, and comprehensive audit logging.

In one approved automated response scenario, the system identified a suspicious .git file on an endpoint. The AI categorized the file as unauthorized code, quarantined it, and suspended the associated software to prevent execution. While this demonstrated capable proactive prevention, Mittal noted the AI also generated new types of false positive alerts for the team to process. Scaling these tools across sprawling OT and legacy environments will require continuous tuning.

Testing autonomous control in a financial SOC

Gupta’s financial organization operates under strict state-level regulatory oversight, including privacy mandates from California and Texas, meaning any automated action carries significant compliance weight. His team found AI highly effective for structured tasks like fraud detection, algorithmic trading, automated underwriting, and updating deterministic playbooks.

However, when applied directly to SOC workflows, fully autonomous AI proved unreliable. During a two-week test in a non-production environment where the AI was granted full control over alert management, the model struggled to process the reality of SOC data. Encountering incomplete fields, inconsistent identifiers, and ambiguous signals, the system incorrectly removed authorized users from the network.

Consequently, Gupta concluded that while AI excels at summarizing complex data, correlating context, and structuring narratives from multiple security tools, final decisions must remain with human analysts. By shifting documentation and context-gathering tasks to the LLM, the organization saved analysts 10 to 15 hours per week, measurably reducing fatigue and context switching.

Governance and paths forward

The trials offer a practical framework for organizations facing executive pressure to adopt AI technologies. Integrating AI successfully requires a risk-based approach that prioritizes problem clarity, strict access management, and human-in-the-loop validation for high-impact decisions.

Security teams evaluating AI should establish the following controls:

  • Data and access management: Build controls to minimize data exposure and create strict identity protocols for AI tools and APIs.

  • Transparency: Require explainable outputs for any decisions that impact access or compliance.

  • Continuous validation: Actively test systems for unauthorized inputs and treat AI models like any other critical security control by routinely auditing their results.

As organizations adapt to new capabilities, security teams must stay actively engaged in the deployment process to support innovation safely. Summing up the relationship between security initiatives and organizational goals, Mittal advised: "Business drives security. Security doesn't drive the business."