ESG White Paper

Better Together

data comfiguration
By Jon Oltsik, Senior Principal Analyst and ESG Fellow
August 2019

58% of organizations reported that they planned to increase their cybersecurity spending throughout 2019."

Executive Summary

Cybersecurity is business-critical and 58% of organizations reported that they planned to increase their cybersecurity spending throughout 2019.1 Unfortunately, a lot of this spending will go toward addressing existing problems in areas like security analytics and operations. What challenges exist and what should organizations do to address them? This white paper concludes:


Cybersecurity continues to grow more difficult. For example, more than three-quarters (76%) of cybersecurity professionals say that threat detection and response is more difficult today than it was two years ago. Furthermore, survey respondents identified numerous threat detection and response challenges like constant emergency responses, a growing attack surface, and “blind spots” in security monitoring. Many organizations also bemoan the fact that cybersecurity analytics and operations are based upon disconnected point tools and manual processes, limiting the responsiveness and scale of the security operations center (SOC).2


SOAPA represents a new strategic security technology direction. To overcome their cybersecurity challenges, many organizations are consolidating and integrating disparate security analytics and operations tools, by building a security operations and analytics platform architecture (SOAPA). SOAPA aggregates security data and connects analytics engines, acting as a force multiplier for security analytics and operations efficacy and efficiency.


SIEM and UEBA work well together. Many organizations have security information and event management (SIEM) platforms in place as a foundational technology. UEBA can add value to security information and event management (SIEM) in several ways:

  • UEBA is designed to detect unknown threats by monitoring user and entity behavior, spotting anomalous behavior, and then measuring risk associated with these anomalies.

  • UEBA can also act as a pivot point from SIEM to provide user context around SIEM security alerts.

  • Finally, UEBA can streamline security operations by supplementing SIEM with detailed high-fidelity alerts that can be used to accelerate incident response (IR) processes.


CISOs should look for tightly coupled UEBA and SIEM integration—from the data layer to the user interface—
to get the most value out of SOAPA."

The State of Security Analytics and Operations

According to a recent ESG research survey, 76% of organizations believed that threat detection and response management is more difficult today than it was two years ago. Why is this the case? Several reasons:



believe that threat detection and response has grown more difficult because the volume and/or sophistication of cyber-threats has increased. Countering this growing threat requires advanced skills and deep knowledge about cyber-adversaries. Many organizations don’t have these qualifications.



believe that threat detection and response has grown more difficult because the threat detection workload has increased. CISOs often observe that the cybersecurity staff can’t keep up, forcing members to ignore alerts and general security hygiene.



believe that threat detection and response has grown more difficult because of a growing attack surface. Many organizations are embracing digital transformation applications, public cloud computing, SaaS, and IoT devices to drive revenue and cut costs. These IT initiatives greatly expand the cybersecurity team’s purview and once again, they find it difficult to keep up.

As if this weren’t bad enough, cybersecurity professionals report that threat detection and response activities are fraught with many challenges, including (see Figure 1):

Figure 1. Top Seven Threat Detection and Response Challenges

Which of the following best describes the role that data storage technology plays in your organization's IT and business operations? (Percent of respondents, N=356)

Source: Enterprise Strategy Group


Keeping up with emergencies. More than one-third (36%) of respondents say that their cybersecurity team spends most of its time addressing high-priority/emergency issues and not enough time on strategy and process improvement. Organizations engaged in this type of “firefighting” can’t manage the volume of alerts while SOC teams suffer from employee burnout and attrition. This is a losing formula.


Addressing IT scale. Thirty percent of respondents claim that their organization has added new network hosts, applications, and/or users so it is difficult for the cybersecurity team to keep up with the scale of their IT infrastructure. This speaks to the growing attack surface, which shows no signs of abating.


Monitoring blind spots. Thirty percent of respondents report that there is one or several “blind spots” on their network, impacting their ability to detect or respond to threats in a timely manner. As the old management adage goes, “you can’t manage what you can’t measure.” In this case, you can’t respond to threats hidden by blind spots in security monitoring.

It is also worth noting that nearly one-quarter of respondents (24%) say that their organization does not have the tools and processes in place to operationalize threat intelligence, making it difficult to compare on-premises threat detection actions with what’s happening “in the wild.” This is especially troubling, since increasing difficulties associated with threat detection and response are often related to the volume and sophistication of cyber-attacks.

ESG data suggests an alarming trend: many organizations don’t have the people, processes, or technology in place to keep up with threat detection and response requirements. This greatly increases cyber-risk, leaving organizations vulnerable to devastating cyber-attacks and data breaches. Clearly, something must change as soon as possible.


To address security monitoring blind spots and operational complexity, ESG believes that enterprise organizations need a tightly coupled security operations technology architecture where independent analytics and operations tools can share data, cooperate with analytics engines, and automate security operations tasks like incident investigations or system remediation. This exchange and synthesis of sources like log and flow data can centralize analytics and help organizations with thorough threat investigations. ESG calls this a security operations and analytics platform architecture (SOAPA, see Figure 2).


SOAPA features a:


Common distributed data service. All security telemetry is collected, processed, and exposed as a distributed data service. This helps organizations create the right data model and data pipeline to address real-time and historical data analysis needs using sources like point-in-time security events and timelines of actions with network flows. These tools can share data and insights that can act as triggers for setting off workflows.


Software services and integration layer. This layer makes the data accessible to any and all security monitoring tools at the right time and in the right format.


Analytics layer. All types of analytics tools (such as SIEM, UEBA, and TIP) can tap into the data and software services layer for real-time, batch, and retrospective analysis.


Security operations platform layer. SOAPA is intended to deliver high-fidelity alerts through efficient data pipelining and software interoperability. Once these alerts surface, SOC personnel can utilize the security operations platform layer for remediation and incident response with services like trouble ticketing, case management, workflow management, and process automation/orchestration.

SOAPA is designed to address the threat detection and response challenges described previously by replacing disconnected and inefficient security analytics point tools with an interoperable security operations architecture. SOAPA can then present the SOC team with high-fidelity alerts (in other words, true positive, high-priority incidents along with a detailed “breadcrumb trail” of evidence). These high-fidelity alerts can help organizations address the scale and scope of threat detection and response by eliminating cybersecurity “noise,” allowing SOC teams to better prioritize actions, automate tasks, and protect their critical cyber assets.

SIEM and UEBA: Critical SOAPA Analytics Engines

SOAPA can deliver strong value when security analytics systems share data and complement each other by analyzing the same incidents through different lenses. SIEM systems have long been a foundational SOAPA technology, analyzing machine data like security events and log files, and even network data like NetFlow records. Organizations tend to use SIEM for threat detection through event correlation rules, dashboards, and rule sets provided by SIEM vendors and a user community. Flows can also be used for real-time security analytics for detecting anomalous/suspicious network behavior and connections.

As organizations gain experience with SIEM platforms, they develop strong analytics and processes for detecting known cyber-attacks like phishing/social engineering, ransomware, and privilege escalation. Many organizations also utilize SIEM as a security operations platform (SOP), supplementing SIEM with security operations, automation, and response (SOAR) software for tasks like case management and system remediation.

To augment SIEM capabilities, organizations are pivoting SOAPA toward user and entity behavior analytics (UEBA), especially for detection and response of unknown types of threats. In true SOAPA fashion, UEBA builds upon SIEM value by supplementing machine data with user data to determine who did what and when (see Figure 3). UEBA does this by focusing analytics on user activity through data sources like identity and access management (IAM) systems, Active Directory, VPN logs, DLP logs, CASB logs, and SWG logs. Leading UEBA tools enhance event correlation and rule sets with machine learning algorithms for monitoring user behavior.

Figure 3. SOAPA Value Increases with SIEM and UEBA


UEBA is especially useful for:


Detecting insider threats. Insider attacks can be difficult to detect for several reasons. First, insiders may have appropriate credentials to access sensitive data so they can hide their attacks behind these entitlements. Insiders also know the value of sensitive data and where it is stored. Finally, a devious insider can execute a “low and slow” attack, stealing sensitive data slowly over long periods of time. UEBA systems are designed to detect insider attacks by modeling normal behavior and then detecting behavioral anomalies. For example, a malicious insider may suddenly access sensitive data during off hours, save data to portable storage media, or log into a system from a new location. UEBA can compare all these actions to normal behavior, alert security analysts to anomalies, and back alerts with supporting evidence. Aside from a user’s historical behavior, UEBA also looks for anomalous patterns by comparing user behavior within peer groups. When a single system administrator logs onto the source code repository server from her home computer, an action that no other group member has ever undertaken, UEBA can identify this anomaly, alert the SOC team, and look for other related suspicious user activities.


Providing new insights. UEBA supplements SIEM by providing contextual visualization of security incidents, pivoting off individual users and machines. For example, a SIEM correlation rule may indicate sensitive data exfiltration from a series of IP addresses, a serious security incident. Armed with this information, a SOC analyst can pivot to UEBA to get detailed information beyond an IP address, focused on user and system activity. In this case, an analyst may discover that a terminated employee continued to log onto the network, steal sensitive data, and use email accounts for exfiltration. UEBA can discover this malicious activity by linking all accounts and activities back to a single user.


Risk scoring. When SIEMs and other tools detect security incidents, it is often up to the SOC team to determine what to prioritize. Once again, UEBA can help. UEBA tracks individual anomalous indicators over time, assigns a risk assessment to each, and then combines them to determine an overall risk score. For example, a DLP system may trigger an alert for an email violation when a user sends sensitive data to their personal email account. UEBA sees a bigger picture, including patterns of system access, similar emails sent, frequent policy violations (by the same user), and a spike in email activity unseen within a similar group of users. When a history of anomalies is evident, UEBA generates a high-risk score, prompting the SOC team to prioritize further investigations.


Producing high-fidelity alerts for streamlining investigations. Sound SIEM rules are great for alerting on individual security incidents. When these occur, analysts can pivot to UEBA to triangulate incidents from a user perspective. This will give them a visual context of who did what and when, providing a historical perspective on what happened before and after the security incident occurred. Leading UEBAs enhance this timeline with rich details around system, network, and user activities. Rather than focus on technical minutiae like examining DHCP logs to find out who had the lease for a specific IP address two weeks ago Wednesday, security analysts can dig into the right user-based historical data, helping accelerate and focus security investigations and incident response.

In summary, UEBA can enhance SIEM by providing context, historical details, and a user/system perspective to security alerts. Armed with the right rule sets and dashboards, security information and event management systems can detect known threats like brute force password cracking that follow a repetitive pattern. Alternatively, UEBA is designed to detect unknown threats by aggregating together sequences of malicious activities into high-fidelity risk scored alerts. In this way, UEBA can help organizations improve known and unknown threat detection to address the threat detection and response challenges described previously.

SIEM and UEBA: Better Together

SIEM and UEBA tools can be deployed independently so organizations may be tempted to look for best-of-breed systems in each area. Yes, this strategy can work, but ESG believes that leading single vendor solutions can deliver tighter integration and faster time-to-value than a more piecemeal architecture.

IBM has a long history as a leading SIEM provider with QRadar. As part of its security analytics and operations products, IBM also offers QRadar User Behavior Analytics (UBA) as a (free) add-on module to its core SIEM. QRadar UBA can complement SIEM functionality with:


Consolidated security telemetry. As described previously, the foundational layer of SOAPA is a common distributed data service that consolidates security telemetry for the analytics engines. IBM QRadar provides this consolidated data service across SIEM and UEBA by aggregating log data, flow data, threat intelligence, identity and access management data, cloud data, and application data into a common data management tier. This consolidated data can help organizations avoid redundant data collection and time-consuming data management. Alternatively, security analysts will have a single repository providing a wide-angle view of all security data for investigations.


Out-of-the-box analytics. Users can look to IBM to provide out-of-the-box analytics for both QRadar and QRadar UBA to detect cyber-attacks like insider threats, ransomware, and phishing. Investigative routines can then pivot from SIEM to UEBA to accelerate and streamline investigations. It is worth noting that IBM Resilient extends the IBM SOAPA as a security operations platform for process management, automation, and orchestration, complementing SIEM and UEBA.


A single familiar UI. UEBA provides a pivot point for SIEM analysts to examine security incidents through a visual and contextual view centered on users and devices. By tightly coupling its products, security analysts can navigate across SIEM and UEBA through a familiar interface of dashboards, reports, and graphics. This should obviate the need for UEBA-specific training and accelerate time-to-value as organizations add UEBA to their existing QRadar infrastructure.

The Bigger Truth

ESG research data indicates that many organizations face constant struggles around security analytics and operations. They have too many independent tools, too many manual processes, and not enough skilled people to glue everything together. This is a recipe for cybersecurity disaster.

Large organizations tend to anchor their SOCs with SIEM systems, which is a good start, but ESG believes that security analytics and operations requires a broader architectural solution and that SIEM and SOAPA are two different things. SIEM is an individual security analytics product while SOAPA is an architecture that can be used to consolidate security telemetry and integrate multiple security analytics engines. In this way, SOAPA is designed to combine individual security analytics and operations tools into a system where the whole is greater than the sum of its parts.

IBM is one of the few vendors that not only understand SOAPA conceptually but also deliver SOAPA today. Case in point, IBM can provide a SIEM/UEBA solution with the combination of QRadar and QRadar UBA. Furthermore, these individual solutions are tightly integrated with a unified data management service, cooperative analytics engines, a single UI, and a security operations platform (Resilient). In this way, IBM can help organizations build SOAPA while accelerating the benefits associated with SIEM/UEBA integration.


IBM QRadar User Behavior Analytics


Source: ESG Master Survey Results, 2019 Technology Spending Intentions Survey, Mar 2019.

Source: ESG Master Survey Results, The Threat Detection and Response Landscape, Apr 2019. All ESG research references and charts in this white paper have been taken from this master survey results set, unless otherwise noted.

This ESG White Paper was commissioned by IBM and is distributed under license from ESG.

All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at 508.482.0188.