What's new in IBM Security Guardium V10
Analyze, adapt, and protect
Editor's note: For V10.1.2 updates and enhancements, refer to the newer article "New and enhanced Guardium Outlier Detection."
A major release - and here's why!
The world of data protection is more complex today than it's ever been before. Data collection continues to grow and is even more dynamic; because data is so valuable, it moves and evolves as it is used for different purposes from online transactions or web applications, through data marts and warehouses for business reporting, and into big data platforms for in-depth analysis or long-term storage.
Data is used in a variety of ways, thus more and more users and developers are touching this data. Not all think about security, nor do they have the coding skills to build in security from the ground up. Thus, it is even more important that the layer of data protection closest to the data is up to the job in this new and riskier environment.
With Version 10, IBM Security Guardium is officially renamed to be part of the IBM Security portfolio, which reflects the importance that security organizations worldwide are placing on data protection. More than a renaming, IBM Security Guardium takes a major step forward with intelligence and automation to safeguard data. An emphasis is placed on capabilities to make the solution more adaptable and easier to use in a dynamic IT environment while it continues to evolve its analysis and intelligence capabilities and breadth of coverage, such as protection for file system data.
Figure 1. Guardium data protection mission
This article is a technical overview of the new features and changes in Version 10, including enhancements that were delivered as part of 10.1 that became generally available on June 3, 2016. Except for the description of a new offering, file activity monitoring, most of what is described here targets experienced Guardium users who have some understanding of the existing capabilities. As usual, please monitor updates via release notes to learn of additional changes that are delivered via the service stream or minor releases. Keep up to date with Guardium by joining the Guardium community on IBM developerWorks.
Common platform enhancements
Guardium solutions for databases, data warehouses, file systems, and applications are built on a common platform. This section describes the enhancements in the common platform:
- Improved user experience
- Enhanced analysis and investigative capabilities
- Classification enhancements
- Compliance accelerators now part of base platform
- Support for CIDR notation
- New backup location option: SoftLayer Object Storage
- Appliance updates
Improved user experience
A major focus of Version 10 has been to modernize and simplify the user interface, an effort that will continue over subsequent releases. This section covers the following enhancements:
- The first thing you'll see...
- A scenario-based task flow to discover and protect sensitive data
- Easy UI customization
- Simplified report management and discovery
- Streamlined audit process builder
- Service status dashboard and consolidated support and maintenance view
- Find and manage datasources more easily
- Ease of use improvements in managed environments
The first thing you'll see...
When you first install Guardium, you'll notice that it has a new look and feel, powered by modern technology. Let's walk through some of the changes you'll see.
The banner is a powerful control center with alerts, to-dos, and an enhanced search bar. The UI search bar will be your best friend in helping you find a tool or report quickly by name.
Figure 2. The powerful banner enables navigation and search
The notifications will let you know about important things, such as:
- A link to the page to apply the license key and accept the license terms if a license has not been installed yet
- Insufficient memory or processors or disk space
- Expiring certificates
Figure 3. Example notification for expiring certificates
- Whether software updates are available, with a direct link to the IBM
support page where you can download them, as shown in the following
Figure 4. Notification indicating updates are available
The navigation on the left is now simplified and normalized across both administrator and user roles:
Figure 5. Navigational elements are common across roles
The help system has also modernized. It includes features you may already be familiar with from the IBM Knowledge Center on ibm.com, including improved navigation, more granular print options, and greatly enhanced search capabilities. You can access the help system from the banner or from the question mark icons on the individual pages in the UI.
Figure 6. Accessing the help system from the Guardium banner
A scenario-based task flow to discover and protect sensitive data
An example of the direction that the Guardium UI is taking can be seen in a new task flow that takes you end to end through a guided workflow that goes from sensitive data discovery, to data protection (defining security policies), to compliance (defining audit process); all without requiring users to jump from place to place in the user interface.
If you go through the entire workflow, relevant artifacts are created, such as a classification policy, an audit process to schedule the classification, and even a security policy with the relevant access rules to protect discovered sensitive data.
Figure 7. A completed sensitive data discovery process
Easy UI customization
The process to customize the user interface and manage permissions for different roles is dramatically simplified in Version 10. Everything is in one central location and uses a simple "slushbucket" approach.
For example, if you want to create a very simple interface with only a few read-only reports for a particular auditor, it can be done quickly and easily. The Guardium access manager creates a new role called "Myfavoriteauditor." For the role, Guardium access manager goes to Manage Permissions and gives very limited permissions to the user as shown in the following figure, which includes Audit Process To-Do List, Report Builder, and Results Viewing.
Figure 8. Manage permissions for roles
Then the access manager goes to Customize Navigation Menu for that role and specifies which reports "Myfavoriteauditor" can see.
Figure 9. Customize the navigation menu for roles
Now when users are assigned the "Myfavoriteauditor" role, they will only see the simplified navigation when they log in.
Figure 10. The Guardium user interface is easily customizable
In 10.1, the following enhancements to permissions management were added:
- The ability to grant permissions on specific reports.
Now you can specify that certain roles can only see certain reports. The example below shows that the person with the role "limited" will only see the five reports specified and cannot add any other reports to their dashboards.
Figure 11. Grant permissions on specific reports
- More granular permission management of administration capabilities.
To improve control over administration capabilities, the Administration Console application ID was removed in V10.1 and replaced with 19 finer granularity ones. This enables administrators to pick and choose more carefully which roles should have permissions to do things such as installing policies and creating inspection engines.
Simplified report management and discovery
Guardium includes hundreds of built-in reports and a flexible reporting capability to allow you to create as many custom reports as you need. The sheer number of reports can make finding your own important reports a bit more challenging.
Version 10 introduces the concept of "My Dashboards." A dashboard is a user-personalized space in which you can drop reports and organize reports for easy access. Each user can name the dashboards and create as many dashboards as needed.
The following figure shows My Dashboards in the navigation menu. You can see that this user already created several dashboards to help organize reports by topic.
Figure 12. Navigating to report dashboards
The following figure shows a particularly colorful dashboard and highlights some of the widgets, including customizing the look of an existing graphical report, changing the runtime parameters, and marking reports as favorites.
Figure 13. An example of a dashboard
By using favorites it enables you to filter reports in audit processes or when you are creating new dashboards so that you don't need to scroll down through hundreds of reports or devise your own naming scheme to ensure that your reports filter to the top of the list.
When you add a report to a dashboard, you can find them easily by name by typing in the first few characters in a field that requires selection from a list.
Streamlined audit process builder
Audit processes are used in Guardium to automate many periodic tasks such as reporting and scanning and to send results to designated receiver roles for review or sign off. The builder is revamped to lead you step by step through the process and includes smarter defaults and usability features.
For example, when creating a report task you can filter your favorites. For reports, security assessments, and classifier tasks, you can use typeahead to locate reports by name.
Figure 14. New audit process builder has many usability features
Service status dashboard and consolidated support and maintenance view
Administrators will love this new central location to see the status of Guardium services. And it provides a one-stop launchpad to where you need to go to configure the service.
Figure 15. Centralized Guardium services dashboard
Support and diagnostic tools are now all consolidated under Manage>Maintenance to help you maintain and troubleshoot your system.
Figure 16. Consolidated maintenance view
Find and manage datasources more easily
Enterprises with hundreds and thousands of datasources will particularly benefit from this new datasource management interface. You can filter and sort this table quickly to find datasources by type, name, or other typical characteristics. You can create, edit, copy, and delete datasources from the new datasource management interface as well.
To access the new interface, navigate to Setup > Tools and Views > Datasource Definitions.
Figure 17. Finding and managing datasources
Ease of use improvements in managed environments
Guardium has a federated architecture to support scalability and manageability. This architecture relies on a Central Manager to provide administrators with a single console for management and control. In 10.1, there are several enhancements to make the Guardium administrator's job easier and to improve overall effectiveness and efficiency.
At-a-glance deployment health
In 10.1, administrators are able to see at a glance if there are issues in their managed environment and address them before they get worse.
The following screenshot is an example of this deployment health view, which you can see from a Central Manager by navigating to Manage > System View > Deployment Health.
Figure 18. Central Management Health View
When configured properly, the view will show parent-child relationships between nodes based on data import and export configurations. Hovering on a node will provide more details on the particular health diagnosis based on connectivity, unit utilization, and aggregation.
As an example, the following collector details show that this unit is experiencing growing issues with disk usage and does not have an aggregation that is scheduled. By addressing this issue quickly, administrators can fix the issue before it becomes more serious.
Figure 19. Detailed health status and actionable links
A tabular view of health also exists, as shown in the following image. This tabular view provides a flat view of all details, has sortable columns, and it can be exported to CSV format as well.
Figure 20. Tabular health view
More flexible configuration pushdown
Guardium is designed to be centrally managed, including the ability to push configuration information to managed units from the single Central Manager console without requiring changes to the CM configuration itself. Prior to V10.1, this process could be complex and was not as flexible as enterprises need it to be.
Now, using a simple wizard, administrators can create configuration profiles for certain types of configurations and specify which managed unit groups should receive those profiles. The following screenshot shows the configuration profile builder.
Figure 21. Building configuration profiles for distribution
In Version 10.1, the following configuration types are supported:
- Unit utilization schedule
- Policy installation
- Data import
- Data export
- IP-to-host aliasing
In addition, a distribution run log report will alert administrators if the configuration was distributed and received correctly by the managed units.
Configuration profiles can be imported and exported.
Simplified management of correlation alerts in a managed environment
Correlation alerts are a powerful capability in Guardium that allow you to build alerts that can fire based on database events that reach a certain threshold over time. Correlation alerts are also used frequently for system events to help administrators keep an eye on system health. Management of these alerts is much easier in 10.1 because you can set separate alerts for different managed units (MUs) directly from the Central Manager. If you didn't want a particular alert to run on a managed unit, you would need to disable it on that MU.
The following figure shows that you can create an alert that is activated on any particular unit or group of units. There is also a new display from Anomaly Detection that shows which alerts are deployed and where. (On a non-CM, the display would only show the alerts for this managed unit.)
Figure 22. Alert builder and display
Enhanced analysis and investigative capabilities
Guardium has been steadily delivering enhancements to help analysts visualize and explore data. Guardium 10.1 delivers improved investigation capabilities for both database and file activity monitoring. We do not cover outlier detection, which is thoroughly covered in another developerWorks article. (See Related topics.) The following capabilities are described here:
- Quick Search for the enterprise
- Enhanced investigation dashboards
- Executive overview dashboard for data protection (10.1)
- Attack threat detection analytics and diagnostics (new in V10.1) (This is covered in the Database Activity Monitoring section of this article.)
Quick Search for the enterprise
This capability was introduced in a 9.5 patch, but it's worth revisiting. Previously, Quick Search was only available on a per-collector basis, requiring you to know which collector had the data you wanted to search. Quick Search for Enterprise simplifies the identification of specific data by extending the existing Quick Search interface to support searches across a Guardium environment from a Central Manager, or potentially from any Guardium machine within that environment.
Each collector in a managed environment indexes its own data. There are three "modes" of Quick Search for Enterprise:
- CM only:
- This is the default. Search will be enabled such that search queries can be submitted from the Central Manager and return search results from all enabled collectors. Search can also be initiated from managed units but will return only local results.
- All machines:
- This enables users who are logged in to any collector, CM, or aggregator to submit search queries and get results from all collectors. Note that this mode can result in slower search results.
- Local only:
- This mode limits search queries to the local machine where the search is submitted; it is not possible to get search results from other machines in the Guardium environment.
Use the Guardium API
set_enterprise_search_options to set
Enhanced investigation dashboards
Investigation dashboards evolved since they were first introduced. In V10.1, investigation dashboards were enhanced to include new chart types and additional capabilities such as saving and sharing filters.
In addition, Quick Search data is included on the dashboards so you no longer need to toggle between charts and Quick Search for your investigations.
To access investigation dashboards, there are two options:
- Navigate to Investigate > Search for Data Activity (or Search for File Activity if you have Activity Monitoring for Files).
- Click the Data or Files search drop-down list in the user interface banner.
Investigation dashboards provide a visual and interactive way for analysts to interact with audit data for both file and database activity. Click on an area of interest and the dashboard will be dynamically filtered so that you can hone in on an area of investigation. You can also create and save multiple dashboards to support different investigative use cases and saving filters that can be used by other users.
There are two demos of using the investigation dashboards for specific scenarios on the Guardium YouTube channel. Links are included in the Related topics.
The following dashboard shows one of the system predefined dashboards included in 10.1
Figure 23. Investigation dashboard for interactive analysis (enhanced in 10.1)
Collaboration by using dashboards
Dashboards and their filters encourage collaboration among users in an organization and with others in the community. Filters can be private or shared with others in your organization, simply by specifying which roles are able to use a filter that you save. Sharing filters saves time because one person can define filters for everyone, and you can also share filters to share a specific investigation case. (You can include date/time in the filters so that someone else can see the exact data that you want them to see.)
Dashboards can be shared with other Guardium users by exporting the dashboard definition. Only the dashboard definitions are encrypted and exported, not the filters; this means that if you have a dashboard that you configured with a good set of charts for investigating particular incident types, you can share this knowledge with other Guardium users without including actual attack data or revealing filters.
New animation chart
The animation chart that was added in V10 adds an important dimension of time to the Investigation Dashboard. This Investigation Dashboard helps analysts to visualize activity behavior over time by using data in motion. This chart uses animated bubbles to represent activity over the last 48 hours (at most). The data is "auto-played," where each frame is an hour in time and can be paused, much as you would when watching any video.
All four dimensions that are used in the chart are configurable: the bubbles, their sizes, as well as the X- and Y-axes. For example, a bubble can be defined as a DB user, the location of its area in relation to the number of client IP addresses, its horizontal position to ACCESS activity, and its vertical position to the number of ERRORS, as shown in the following image.
Figure 24. Animation chart from investigation dashboard
This view supports the ability to drill down; clicking a bubble adds the selected data elements to the filters, and all charts are filtered accordingly.
Executive overview dashboard for data protection (10.1)
Guardium Version 10.1 introduces a new interactive dashboard designed specifically for display in security operations centers or for security executives. This new dashboard (shown in the following image) provides aggregated information that provides CISOs with a snapshot view of their current governance, risk, and compliance stance with the following information:
- Activities, errors, and violations (also shown in animated bubble graph)
- Outlier summary (rolled up outlier information)
- Numbers of DB users, OS users, and client IPs
- Risk, based on Vulnerability Assessment tests
- Compliance, based on open to-do tasks
Figure 25. Data Protection Dashboard provides analysis and visualization for governance, risk, and compliance
This dashboard is designed to stay open for long periods of time. The chart is also interactive so you can apply search filters, but the primary goal for this chart is to give management and executives a snapshot of their current posture and delegate followup tasks to administrators and analysts.
There is a demo of this dashboard on the Guardium YouTube channel. See Related topics for the link.
In addition to the incorporation of classification into an overall workflow as described above, the following enhancements are also included:
- Better controlling false positives by using "excluded groups" for
schema, table, and table/column.
Previously, it was a complex process to set up Guardium to ignore false positive results for future classification scans. Now when you review classifier results, you can easily add false positive results to an exclusion group as shown in the following image, and add that group to the classification policy to ensure that those results are ignored in future scans.
Figure 26. Easily add schemas, tables, or columns to exclusion groups
- New option for reducing the granularity of data results.
Some organizations might want to do a survey of their data to discover which tables and columns have sensitive data without necessarily needing to find every type of sensitive data in that column. A new option, One match per column, means that as soon as the classifier finds a match for that column, it will ignore that column as it continues its processing.
Figure 27. New option to customize classifier behavior
The following table summarizes the classifier processing options when a match is found for this rule.
Table 1. Classifier processing options
Continue on match One match per column Granularity of result No N/A Table. The classifier will stop processing after the first hit in the table. Yes Yes Table and column. The classifier will record the first hit for any given column and ignore it thereafter for subsequent rules. Yes No Detailed. The classifier will record hits for all columns for all rules.
- Improved caching logic improves performance improvements for any classification policy that has multiple rules.
- Performance improvements for Teradata and Oracle classification in some circumstances. Optimizations in this version reduces elapsed time for random sampling, especially in those cases in which other processes are running, or in which table statistics do not exist.
- New CLI commands to allow greater control of timeout values:
- To control timeout for internal queries to gather the number
of rows in a table that was used for random searches, use the
following CLI command:
store timeout classifier count_query <n>
where <n> is the number of seconds (the default is 180 seconds). If the query times out, the classifier reverts to a sequential scan.
- To control timeout for the query to return the data for the
sample size, use the following CLI command:
store timeout classifier sample_query <n>
where <n> is the number of seconds (default is 180 seconds).
- To control timeout for internal queries to gather the number of rows in a table that was used for random searches, use the following CLI command:
Compliance accelerators now part of base platform
Before V10, the compliance accelerators (PCI, SOX, Basel II, and Data Privacy) had to be installed by using separate patches. Now they are part of the base product offering and can be added to the user interface by configuring users with any of the corresponding roles (PCI, SOX, and so on). The following figure shows that the Guardium access manager is giving the user, Kathy Zeidenstein, the PCI and SOX roles. When Kathy logs in to Guardium, she sees the Accelerators navigation menu and can see the content for both accelerators.
Figure 28. Compliance accelerators are easy to configure
Support for CIDR notation
In 1993, classless interdomain routing (CIDR) was created and along with it came CIDR notation. CIDR notation is a syntax for specifying IP addresses and their associated routing prefix. It appends a slash character to the address and the decimal number of leading bits of the routing prefix. For example, 192.168.2.0/24 for IPv4, and 2001:db8::/32 for IPv6 (per Wikipedia.) Before Guardium V10, CIDR notation was not supported.
To support CIDR and to stage support for IPv6 in a later release, Guardium mask fields will now accept a prefix number or a mask. When the input is shown later, it is displayed as a prefix. Here is an example of an input field you might see in Guardium, such as in Policy Builder or Access Map Builder.
Figure 29. CIDR notation can be used in the UI
In V10, you can enter
220.127.116.11 in the first field with a
23 in the second field, which will be interpreted as
255.255.254.0 for the second field is how it used to
be done in previous Guardium versions. This is still allowed, but now when
the screen is updated it will show
23 in the second
New backup location option: SoftLayer Object Storage
Long-term storage is a critical consideration for satisfying audit requirements that might require storage of audit data for up to seven years. The ability to archive and back up to the cloud gives you another option for storage off premises.
In addition, backing up the configuration of Guardium appliances to the cloud is useful for maintaining a disaster recovery environment so that if a local data center has a failure, you can restore the configuration of the appliance from the image that is stored in the cloud.
Guardium now supports SoftLayer® Object Storage as a repository for both audit data and configurations, whether your Guardium system is in a local data center or in the cloud. SoftLayer Object Storage provides self-healing storage for massive amounts of data. There are object storage centers around the world so you can avoid issues of moving sensitive data across country boundaries.
Figure 30. Store backups on IBM SoftLayer for both on-premises and cloud-based Guardium systems
Job scheduling dependency management
The Guardium Collector has many tasks, such as installing policies that are running audit processes and updating groups that are scheduled to run periodically. We'll call these processes and tasks "jobs." In V10, Guardium can detect jobs that are prerequisites for a particularly marked job and run those prerequisite jobs in order before it runs the marked job; this can help avoid schedule collisions or the use of inaccurate or out-of-date data in the execution of the marked job.
This capability can be activated in the Audit Process Builder and in the Policy Installation scheduler by selecting the checkbox Auto run dependent jobs. The following screenshot shows this field in the Audit Process Builder.
Figure 31. Select Auto run dependent jobs to manage job dependencies for this job
The hardware configurations for the appliances are largely unchanged. However, the Guardium system is now based on the hardened version of Red Hat Linux® 6.5 and is only available as a 64-bit platform. Upgrade from a 32-bit platform is not supported. You must rebuild the appliance with a V10 ISO and then apply GPU 100 to get to 10.1.
Another change is the enforcement of what was previously only a recommendation that a Central Manager (CM) to be configured on an aggregate machine and not on a collector.
Finally, a new hypervisor environment, Hyper-V, is now supported in 10.1
Activity Monitoring for Files - New product offering
Activity Monitoring for Files is a new, separately orderable offering in the Guardium portfolio that is designed to provide security and protection for sensitive file data, such as critical intellectual property documents - this includes source code, financial reports, configuration files, and more.
Guardium Activity Monitoring for Files is built on the Guardium common platform and can be used with the same appliance as Guardium Activity Monitoring for Databases or as a stand-alone appliance.
Why do you need activity monitoring for files?
There are many situations in which activity monitoring for files is required, including:
- Critical application files can be accessed, modified, or even destroyed through back-end access to the application or database server
- Requirement to protect files that contain Personally Identifiable Information (PII) or proprietary information while not impacting day-to-day business
- Requirement to block back-end access to documents managed by your application
It's also very important that monitoring and control not impact business operations.
Figure 32. Activity monitoring for files can protect against inappropriate access to application files or sensitive data
Overview of file activity monitoring
File activity monitoring is similar to database activity monitoring in many respects. In both cases, you discover the sensitive data on your servers and configure policies to create rules about data access and actions to be taken when rules are met.
File activity monitoring consists of the following capabilities:
- Discovery to inventory files and metadata.
- Classification to crawl through the files to look for potentially sensitive data, such as credit card information or PII.
- Monitoring, which can be used without discovery and classification, to monitor access to files and, based on policy rules, audit and alert on inappropriate access or even block access to the files to prevent data leakage.
Figure 33. Capabilities of file activity monitoring
File monitoring is supported for a wide variety of Linux platforms (including Power® and S/390® platforms), AIX®, and Windows. Discovery and classification platform support is less extensive.
In 10.1, discovery support was extended to include AIX 6.1 and 7.1. This does not include classification (deep analysis).
The figure below is a high-level view of the architecture to configure file-activity monitoring capabilities.
Figure 34. High-level architecture of file activity monitoring
The file system monitoring agent is included in the same bundle as regular database S-TAP® and is distinguished in the Guardium UI with a :FAM suffix appended to the S-TAP host name. The discovery agent, sometimes called the "crawler," is distinguished with the FAM_Agent suffix as shown here.
Figure 35. File activity agents
Discovering files and sensitive data within files
A FAM discovery and classification agent is required if you want to conduct file discovery and classification. It is not required for monitoring activity, but it does require that an S-TAP is already installed on the file server.
Use Guardium Installation Manager to configure this agent to conduct a file discovery (a basic scan) and optional classification. To minimize impact, the scan runs in offline mode and not in real time. Also, after the initial scan, subsequent scans will only pick up new and changed files.
The basic scan includes Owner, Size, Last Change, and Access Privileges to a user or group.
For classification, you use sets of classifier rules known as decision plans. Default decision plans exist for HIPAA, PCI, SOX, and source code and can be activated and configured through the Guardium Installation Manager (GIM).
You can create your own customized decision plans using IBM Content Classification workbench, a Windows™-based system, which is included with Guardium Activity Monitoring for Files. The following screenshot shows the authoring interface.
Figure 36. Workbench to create customized classification rules
Export the newly authored Decision Plans to the Guardium appliance, and use GIM to push them down to the discovery and classification agent.
Viewing discovery and classification results
You can see entitlements and classification in online reports or in Quick Search. To view entitlements and classification data for files in Quick Search, choose File in the search drop-down list in the banner. This action opens the Quick Search function and displays file data. The FAM Discovery Agent must be configured to scan and send data to Quick Search.
You can even automatically add discovered files to a security policy rule to set up monitoring, alerting, and blocking.
Figure 37. Classification and file privilege information in quick search
Classification and file privilege information in quick search
Use security policies to define what actions, if any, that Guardium should take on access to a file or file directory or subdirectories.
An important note if you are familiar with the way Guardium does database activity monitoring: with file monitoring, the rules are pushed to the data source and are evaluated there.
Figure 38. Create a new rule
Actions that you can specify are:
- Alert and Audit
- Send an alert to a designated receiver and log the event.
- Audit only
- Log the event.
- Block, Log as Violation, and Audit
- This blocks access to the file, logs it as a policy violation, and logs the event. It also sends an alert to a designated receiver.
- Ignore this event. This is useful for trusted traffic or applications to reduce the amount of traffic sent to the Guardium system.
- Log as violation and Audit-
- Log this as a policy violation and log the event.
There are several predefined reports that you can use and modify for your own auditing requirements, including a count of activity per client, server, or user.
What's new in activity monitoring for databases
Activity monitoring for databases is the flagship offering in the Guardium portfolio, and provides a wide breadth of capabilities to help you manage your data protection needs, including data discovery and classification, auditing, alerting, compliance workflow, and advanced analytics for proactive detection of threats. Guardium supports a wide variety of relational systems, NoSQL systems, Hadoop-based systems, and data warehouses.
The significant investment in usability and other common platform features are described at the beginning of this article; this section focuses on specific enhancements for Data Activity Monitoring users, including the following:
- Attack threat detection analytics and diagnostics (new in V10.1)
- Enterprise readiness and adaptibility, which includes enterprise load balancing, enhanced instance discovery, and GIM improvements
- Policy-based fine-grained access control
- Using MongoDB as an audit repository
- S-TAP performance and throughput enhancements
- Database platform enhancements
- New S-TAP platforms
Attack threat detection analytics and diagnostics (new in V10.1)
There are a wide variety of database attack mechanisms. In Version 10.1, Guardium includes specialized threat detection analytics that scan and analyze audited data to detect symptoms that may indicate the following database attacks:
- SQL injection, which is usually an attack on the database through improperly coded web applications. These include classic injection techniques such as leveraging the use of escape characters in input fields to inject SQL, blind injection techniques that use automation to piece together information piece by piece based on the logical result of query manipulation, and other SQL injection techniques.
- Malicious stored procedures, such as might be caused by a disgruntled DBA that decides to use a stored procedure to disguise a drop of an important table or extract the contents of a table
Unlike some solutions, Guardium does not rely on comparison against a dictionary of attack signatures, which can change endlessly. Instead, Guardium automates database threat analysis capabilities into the product by analyzing audit data activity, exceptions, and outlier data over time for specific patterns of events that can indicate a SQL injection attack or malicious stored procedure.
Guardium can detect activities that might be suspicious, such as the creation of a stored procedure with a DROP statement with sensitive objects and a DROP verb or SQL exceptions caused by missing objects. Outliers detect that the DBA did something unusual after it was dormant for a while in modifying the stored procedure. Because of the length of time of the attack, and the various different possible events and users, it can be difficult to tie these events together to see the whole picture of the threat. Threat analytics introduces a new, higher level of detection. It tracks the suspicious events over time and correlates them together to create a clearer picture of potential risks.
Guardium analyzes symptoms over time, correlates them, and assigns a total score per attack type. If the total score indicates a likely attack, that result becomes a case. Cases are externalized in case reports, which are delivered to analysts by using predefined audit processes that you can tailor. There is one case report for each attack type.
The case report includes a link from each case to the following:
- The detailed symptom report for this specific case. An example is
Figure 39. Link to case symptoms directly from the case report
- The investigation dashboard for this specific case. General
investigation dashboards were covered in Enhanced analysis and investigative
capabilities . These dashboards are tailored for threat detection
attack types and are known as threat diagnostic dashboards. Each
dashboard launches with the filters set based on best practices for
starting an investigation. In addition, specific charts on the
dashboard might include reference data, which means data
outside of the dashboard filter is included to allow for comparison.
The following screenshot is an example of a threat diagnostic
dashboard for a suspected malicious stored procedure case.
Figure 40. Link to threat diagnostic dashboard directly from the case report
Note: Currently, threat detection diagnostics are supported only for Oracle, MySQL, and Db2 distributed databases.
Enterprise readiness and adaptibility
Guardium includes capabilities to help automate the management of the Guardium infrastructure. Some enterprise capabilities were covered in the common platform enhancements, such as the new Central Manager deployment health. This section describes the following capabilities:
- Full-featured enterprise load-balancing using Central Manager
- Enhanced instance discovery
- Guardium Installation Manager (GIM) improvements
- Support for OS user and database name in tuple group (10.1)
- Link inspection engine identifiers with logged activity (10.1)
Full-featured enterprise load-balancing using Central Manager
Dynamic load balancing is available in centrally managed environments and reduces the workload on Guardium administrators by automating several tasks that previously required manual tracking and intervention. Dynamic load balancing:
- Eliminates the need to manually evaluate the load of managed units before assigning those managed units to an S-TAP agent.
- Eliminates the need to define fail-over managed units as part of post-installation S-TAP configuration, because the load balancer dynamically manages failover scenarios.
- Eliminates the need to manually relocate S-TAP agents from loaded managed units to less-loaded managed units.
Restrictions: Dynamic load balancing is not supported for z/OS and IBM i S-TAPs.
How enterprise load balancing works
The dynamic load balancer is an application that runs only in the Central
Manager (CM). It requires no special configuration to run. The load
balancer application is enabled on the Central Manager by setting
LOAD_BALANCER_ENABLED=1. It will only affect the behavior of
those S-TAPs that are installed with a value for the
load_balancer_IP (or STAP_LOAD_BALANCER_IP if you are using
10.1 update: In V10.1, the
can be any managed unit. Previously, it had to be the Central Manager IP.
Information that is required for load balancing will be forwarded to the
Central Manager by using a proxy on the designated managed unit. This
enhancement removes the requirement to open a port between the database
server and the Central Manager.
Important:To use the proxy support, you must use a Version 10.1 S-TAP. This capability is not supported in any S-TAP prior to V10.1
The dynamic load balancer performs load collection periodically, which entails getting a snapshot of current activity load for all active managed units and storing it in a load map. This load collection does not affect other activity on the CM. You can specify the load collection by using a fixed interval or dynamical collection. Dynamic collection is the default and recommended setting. With dynamic collection, intervals will be determined by the number of managed units (one additional hour for every 10 managed units). Dynamic intervals will guarantee a more accurate load map without loading the CM with unnecessary load collections.
Figure 41. Central Manager collects snapshots of the current loads on the collectors
If something changes for a particular managed unit that affects its load, such as a reduction or increase in the number of S-TAPs that are connected to it, the load balancer will be notified through a change tracker on the MU and updated information will be sent to the load balancer.
Once the load balancer has the load map, it can make informed decisions about which collectors are best suited to failover, new allocations, or for rebalancing of S-TAPs. (Note that rebalancing can only happen after a full load collection and is controllable via a load-balancer configuration parameter.)
Using groups to manage and control the load balancing environment
It's likely that you have different "zones" for different groupings of database servers/S-TAPs and managed units. You can use the following two types of groups to set up your environment for load balancing:
- S-TAP groups
- MU groups
Figure 42. Use groups to control and manage load balancing
You can create and associate these groups ahead of time in the Central
Manager interface. The group names are case-sensitive. For the S-TAP
groups, you must specify exactly what you will use to install the S-TAP
itself (either the host name or IP). You can use wildcards in your IP
addresses, such as
You can also specify these groups during S-TAP installation. (The MU group must exist already. If an S-TAP group doesn't exist at installation, Guardium will create it for you.)
Reporting on load balancing events
Administrators can see whenever rebalancing or failover events have occurred using the S-TAP load balancer report.
Figure 43. Load-balancing event report
In 10.1, there is also a load balancing report that shows a snapshot of the current load distribution. Previously, this data was only available by using GRDAPI commands. Find this report by going to Manage > Reports > Unit Utilization > Load Balancer.
Enhanced instance discovery
Guardium, with auto-discovery enabled, gives you the ability to use the power of S-TAP to discover running instances on that server, including the information that you need to automatically populate the inspection engine definitions. V10 makes it much easier by not requiring Java™ technology or any external libraries to accomplish this task.
To enable instance discovery, use the following flags during S-TAP installation:
- Noninteractive install flag:
- GIM installation: set
When installation is completed, S-TAP will be configured with Inspection Engines for all running databases.
To invoke instance discovery after installation, go to Manage > Activity Monitoring > S-TAP Control and select the Send Command icon as shown in the following screenshot. Notice that you can optionally replace all inspection engines in that S-TAP with the newly discovered configurations.
Figure 44. Run database instance-discovery process to find instances on the server
The other option is to review the results in the Discovered Instances
report and invoke the
create_stap_inspection_engine API for
one or more discovered instances.
Figure 45. Create inspection engines directly from Discovered Instances report
Instance discovery is supported for the following database types:
Guardium Installation Manager (GIM) improvements
GIM eases the burden of maintaining Guardium GIM modules that reside on the database server, such as the configuration audit system agent, the S-TAP, the discovery agent for files, and the discovery module for database instances.
Easier deployment of GIM clients
Before V10, whenever a new database server was configured with the GIM client on it, it was required to know the IP address of the Guardium appliance it was connecting to. For organizations that stand up new database servers, this required additional communication between the DBA and the Guardium administrator, thus slowing down the deployment of the database server with Guardium monitoring.
Now, by using remote activation, a database server can be installed without specifying a Guardium IP address, thereby putting the GIM Client in "listener" mode. Any GIM client in "listener" mode can be remotely activated from a collector (Manage > Module Installation > Remote Activation) without requiring additional configuration changes on the database server.
You can also auto-discover any servers that have GIM clients in "listener" mode and then remotely activate any or all of those discovered clients.
In sum, this enhancement enables IT organizations to roll out Guardium on all new servers without requiring further interactions with the Guardium team, which can activate Guardium on the database server on their own.
Improved security by using external certificate authority
Before V10, GIM connections between the database server and the GIM server used Guardium self-signed certificates. With V10, you can now use an external certificate authority to authenticate these connections. It is fully backward compatible with older GIM clients.
GIM client bundles are pre-installed with Guardium self-signed certificates. By default, new installations of GIM clients will attempt to establish secure and authenticated connections with the GIM server over port 8446. You can use your own keys and certificates either by installing the GIM client with the relevant certificate information or by updating it using the GIM GUI or API.
Updating keys/certificates throughout a large site can be a long process. During that time there might be a mismatch between the GIM server and GIM client's keys/certificates. When the GIM client fails to connect to a GIM server (appliance) over port 8446 (secured and authenticated), it will switch to the traditional secured port 8444 and write an event in the GIM Events report.
Support for OS user and database name in tuple group (10.1)
10.1 supports the addition of OS user and database name in tuple-groups used in policy rules. This provides finer granularity for determining which connections would fire a policy rule. The following figure shows the 7-tuple condition in the policy rule builder and examples of what some of the members of the 7-tuple group might look like.
Figure 46. Using 7-tuples in policy and what the group members look like
It's a good idea for this capability to only be used when strictly required as the operation can impact performance on the appliance.
The 7-tuples are only supported for Windows and UNIX® S-TAPs.
Link inspection engine identifiers with logged activity (10.1)
Some organizations would like a reliable and possibly user-defined way to link database activity with the particular inspection engine/database server. Existing methods such as using the port may not always provide the needed information, such as for local connection traffic. Some people may want to use their own identifiers so they can link activity in a meaningful way to downstream systems, such as inventory systems.
In 10.1, the Tap identifier field for the inspection engine (which you can specify in the S-TAP Control panel of the user interface, as shown below, or in the guard_tap.ini file) will be included in the session traffic sent to Guardium. If no identifier is specified, Guardium creates one based on the inspection engine parameters.
Figure 47. Adding an identifier in the inspection engine configuration of S-TAP Control UI
To pull the identifier information into your access domain reports, add the Tap Identifier attribute from the session entity as shown here. (The identifiers in this report are all system defined.)
Figure 48. Adding the inspection engine identifier to your reports
- The Tap identifier capture requires that both the S-TAP and Guardium appliance are on 10.1.
- This is only supported for Windows and UNIX S-TAPs.
- User-defined Tap IDs cannot contain spaces.
Policy-based fine-grained access control
Fine-grained access control is a critical privacy and security feature because it can restrict what data different users can see and under what conditions. A main use case for this capability is dynamic data masking, but you'll see that with Guardium other use cases can be supported.
Fine-grained access control in Guardium provides an extremely powerful and flexible form of access control that allows organizations to quickly address a wide range of security concerns, such as:
- Enforcing security in multitenancy scenarios where multiple users and applications share a single database, but where not all users and applications have access to all data.
- Exposing a database to a production environment for testing purposes without exposing the entire database.
- Rapidly correcting critical security vulnerabilities while permanent solutions are developed at the database or application level.
The Guardium implementation of fine-grained access control has the following characteristics:
- No database changes are required
- No application changes are required
- No network changes or proxies are used
- Masking for local connections is supported (such as Oracle Bequeath)
- Selective connection masking to avoid impact to other applications
- Can be used to prevent access or mask data
With Guardium, you can use its existing policy-based controls over several runtime environmental conditions, such as particular users, client IPs, objects, and commands to determine whether and how data will be masked.
The Guardium implementation of fine-grained access control is known as query rewrite because Guardium can dynamically rewrite the queries between the database client and database server. By rewriting queries, you can:
- Hide data:
- Horizontally by limiting the rows that are returned
- Vertically by limiting the columns that are returned
- Mask field values by replacing values with something else or returning NULL
- Restrict database activity
Table 2. Examples of how SQL commands can be modified to mask data or change behavior
|Access control required...||Action||Original query||Modified query|
|Limit what rows are accesible||Modify or add a WHERE clause|
|Limit what columns are accessible||Modify SELECT list (field)|
|Limit what users can do||Modify command (SELECT, INSERT, UPDATE, DELETE, CREATE...)|
|Limit what users can do||Modify object (table, view, stored procedure...)|
- Oracle (Linux, Unix, and Windows (Windows added in 10.1)
- DB2 (Linux and Unix only)
- Microsoft SQL Server
How it works
Guardium fine-grained access control relies on the following interacting components:
- Query rewrite definitions, which are rules that tell Guardium how to rewrite a query. The rules are based on syntactical elements in a query: objects, verbs (commands), and fields.
- Query rewrite actions in the runtime security policy. Again, a policy rule is how you tell Guardium under which specific conditions to apply any particular query rewrite definition.
Figure 49, below, is a simple example. In this case, a query rewrite definition exists to rewrite a query that contains SELECT * is rewritten to specify only certain columns by name. The runtime conditions (the policy) specify that this rule is to be applied whenever the runtime object is EMPLOYEE and the user is DB2INST.
The flow at runtime is as follows:
- User issues SQL.
- Guardium S-TAP holds the SQL and checks policy rules for conditions.
- If conditions are met, Guardium rewrites the query and sends it to the S-TAP.
- S-TAP releases rewritten query to database server.
- Results are sent back to the user.
Figure 49. Runtime flow for fine-grained access control (query rewrite)
Creating and testing query rewrite definitions
A query rewrite definition is how you specify to Guardium how you want a query to be rewritten at runtime, assuming the policy runtime conditions are true. Simply add that query rewrite definition to the policy action called Query Rewrite: Apply definition.
To create a query definition, navigate to the Query Rewrite Builder (Protect > Security Policies > Query Rewrite Builder). The Query Rewrite Builder includes a space for building the query rewrite definitions and a place to test it against actual SQL statements.
Figure 50. Query rewrite builder interface
select * from customer shown
above is a model query, which serves to make it easier to create
the definitions. The definition is really a set of directions to the
system for rewriting the syntax of a query. So, if we add a WHERE clause
to the model query, this is how it is translated in Guardium as a
"from-to" definition; it is basically saying that when Guardium sees
something at runtime that is defined in query definition, it will change
it FROM something TO something else.
Figure 51. Query rewrite builder translates model query to rules
The definition above is saying that for any query, add a WHERE clause in the top level. This is why it's important that you use policy rules to specify which objects (tables or views) you want this WHERE clause to be added to.
To clarify, the following figure shows an end-to-end view of a query
rewrite definition that we created to redact any rows of data that are
government customers. The security policy invokes this query rewrite
definition whenever the runtime table is
customer and the
user is Joe.
When Joe issues
select * from customer, he will not see any
rows of data for government customers.
Figure 52. End-to-end view of masking rows of data
Reporting on query rewrite activity
There is a new report domain, Query Rewrite, that you can use to create activity reports of when a query was rewritten.
Figure 53. Query rewrite report indicates when and how a query was rewritten at runtime
Using MongoDB as an audit repository
Some organizations would like to write audit data outside of Guardium collector for reasons such as:
- "Post-processing" the audit information for fraud and other analytic analysis.
- Storing information into another data store for long online retention requirements.
In Version 10, it's possible to concurrently write audit data to both the collector database and JSON-formatted files that can be transferred to a MongoDB document database.
Important: Unlike the Guardium collector, the MongoDB database is not a hardened repository. You should carefully restrict and monitor access to the audit data by using Guardium.
How it works
As shown in Figure 54, when properly configured, the parsed audit data is sent simultaneously to the Guardium collector repository and written in JSON format to a file in the following directory: /var/IBM/Guardium/data/auditlog
When a file is ready to be loaded into MongoDB, it will be marked with the
suffix .ready. Use the Guardium API command
to send all ready files
Figure 54. Audit data is sent simultaneously to the Guardium repository and JSON files
The followig screenshot shows an example of selecting a Guardium audit record after it loads into MongoDB. In this example, the audit data from Hadoop (HDFS) access to a text file is named ssn.txt.
Figure 55. Guardium audit data in MongoDB
Configuring the solution
To enable the collection of audit data in JSON format, you must configure the following:
- To enable the feature, as a CLI user, enter the following command and
then restart the inspection core:
store log external state on
Other options you can specify on this command include:
- File_size to specify the maximum size for a given file. The default is 4096MB.
- Flush_period to specify how often to mark files ready. The default is 60 seconds.
- To specify which traffic to collect, use the LOG EXTERNAL rule action. (This action will not appear as an option until after the collector enables.)
To send the data to MongoDB, ensure connectivity between the Guardium collector and the MongoDB server and use this command:
grdapi load_mongodb host="<mongodb-host-name>" port=27017
database="admin" user="> user-name <" password="<mongopassword>"
Currently, only the admin database in MongoDB can be specified.
You can disable MongoDB logging by using the CLI. It is not necessary to remove the action from all policy rules.
S-TAP performance and throughput enhancements
This release has a considerable number of enhancements to improve the performance and throughput capabilities of S-TAP.
Considerations for upgrade: If required, a V10 S-TAP can connect to a V9 collector at certain patch levels. However, it is best to follow the recommended upgrade strategies and get the entire environment on to V10.1 as quickly as possible.
S-TAP multithreading can be used in certain workloads to prevent overrunning buffers in the S-TAP and associated K-TAP. It works by preserving multiple threads from the point of traffic interception to the point at which traffic is sent to the appliance.
To enable S-TAP multi-threading, configure the STAP with
participate_in_load_balancing=4 and the number of connections
you want by using a parameter in 10.1 called
Both parameters are configurable from the guard_tap.ini file and S-TAP
The maximum amount of threads is still five, so you could have one sqlguard section with num_main_thread=5, or two sections with one at 3 and the other at 2. If you specify only one collector, then all traffic goes to the same collector. The total number has to be five or less and this is enforced by the user interface.
Note: When you are using the enterprise load balancer, num_main_thread will always be 1.
When used with pooled connections, the total number of threads to handle data can be up to 50 (that is, 10 * 5).
In 10.1, failover support was added.
Considerations for use: In this configuration, no one
Guardium receives all the data from the S-TAP. The distribution is similar
to the one used when
participate_in_load_balancing is set to
participate_in_load_balancing 1 and 4 are similar, they do
not send the same sessions to the same place, so if you are using 1 and
switch to 4, your sessions will move machines and you'll lose the access
information for those sessions.
Also, as when
participate_in_load_balancing is set to 1,
encrypted and unencrypted A-TAP traffic may not be sent to the same
Make sure to use the same policy on all the connected Guardium systems. If the policies are different, there's no guarantee which policy is in effect on a given session.
64-bit S-TAP binaries (Unix/Linux)
The S-TAP binary for 64-bit platforms is built as a 64-bit binary, which increases the amount of data S-TAP can buffer (approximately 2GB per sqlguard_ip).
New parameters for optimizing performance
S-TAP now ships with the recommended performance parameters:
- This is an existing parameter that is now on by default. When set to 1, the TCP port information is loaded into K-TAP when S-TAP starts up. The result is that K-TAP is no longer dependent on S-TAP to determine which TCP connections need to be monitored, which reduces the likelihood of experiencing database performance degradation if S-TAP becomes slow. For more information about this parameter, see the IBM Redbook® publication, Deployment Guide for InfoSphere Guardium, listed in Related topics.
- ktap_fast _shmem
- Similar to the behavior that already supported Informix®, this is a new parameter that pushes the recommended information for DB2-shared memory configurations to the K-TAP. This means that K-TAP is not dependent on S-TAP to determine which shared memory connections needs to be monitored. In general, don't turn this off.
Database platform enhancements
This table summarizes the new and changed support for database platforms. More details are included below the table for some of the more substantive changes.
Important: Database support is constantly being enhanced and updated via the service stream. Be sure to check the Release Notes for any patches and GPUs. The system requirements page on ibm.com will also reflect the most current support. See Related topics for a link.
Table 3. Summary of enhancements for databases
|Database type||Enhancements||More information|
|Apache Cassandra (and DataStax distribution)||See this developerworks article publication for details on configuring and using Guardium with Cassandra.|
|DB2 for Linux, UNIX and Windows|
|DB2 for i||IBM i enhancements for enterprise-readiness and DB2 for i|
|DB2 for z/OS®||DB2 for z/OS enhancements|
|Data Sets for z/OS||Data sets for z/OS enhancements|
|MongoDB||MongoDB for VA description.|
|SQL Server||Improved handling of Microsoft SQL Server encryption|
Guardium continues to evolve its Hadoop support to make it easier to collect events in real time to provide more targeted reporting capability and more protection capabilities. Version 10.1 also introduces phase 1 of integration with HortonWorks through Apache Ranger.
Blocking and redaction for Impala and Hive
S-GATE TERMINATE and redaction are now supported for Impala and Hive activity. This section describes standard S-TAP, not the approach through Apache Ranger.
Important: Because of the way Hive and Impala traffic is processed in Hadoop, you must do the following in the blocking policy rule:
- Specify the DBTYPE in the blocking (S-GATE ATTACH and S-GATE TERMINATE) policy rules; that is, either Impala or Hive.
- Ensure that ATTACH happens on a combination of user and object/command.
The figure below shows the sequence of events involved in blocking hive traffic, in which a privileged user attempting to access customer data has his/her connection terminated and the attempt is logged as a policy violation.
Figure 56. Hive user is prevented from accessing sensitive data
Redaction is configured by using extrusion rules in Guardium policies. Again, be sure to specify Hive or Impala in the DBTYPE for these rules. Here is an example of a Hive query in which social security and credit card numbers were redacted.
Figure 57. Sensitive data can be redacted for Hive and Impala queries
More targeted inspection engines for data collection
New inspection engines that target specific Hadoop components provide improved parsing and reduce the amount of manual work to report on data elements, such as user name. New options for inspection engine protocol are:
- To parse HiveQL through Beeline. Beeline is a JDBC client based on the
SQLLine CLI — although the JDBC driver communicates with HiveServer2
by using HiveServer2’s Thrift APIs. Before, V10 HiveQL traffic
required you to create computed attributes. Parsing also improved to
parse the traffic just as other relational databases. An example query
and associated activity report is shown below:
Figure 58. HiveQL reporting is improved
- To capture failed logins from Hue requires correct configuration of
the HUE inspection engine based on the Hue metastore you use (MySQL,
Postgres, or Oracle). The following figure shows the inspection engine
when Oracle is used as the Hue metastore and an example exception
report that shows the failed logins.
Figure 59. Hue inspection engine configuration and exception report
- For improved parsing of Impala traffic. Before V10, the user would be
returned as an object. The report example below shows the user in the
Figure 60. Improved reporting of Impala user names
- For improved parsing of WebHDFS traffic, you no longer need to create
a computed attribute to extract the DB user name. See the following
Figure 61. WebHDFS activity report
Improved Hadoop prebuilt reports
Based on feedback from our customers, Guardium now includes built-in reports that more succinctly address required reporting requirements, including the following:
- Privileged User Activity
- Privileged Users Accessing Sensitive Objects
- Sensitive Data Activity
- Unauthorized Yarn Jobs
- User Login
- User Sessions
- Yarn Jobs
Phase 1 of Apache Ranger integration for Hortonworks (V10.1)
Apache Ranger is used in Hortonworks for native access control and auditing. Guardium introduces an integration with Ranger that enables Ranger audit data to be directed to Guardium and also enables Guardium to leverage Ranger access control policies for blocking. The following are benefits that can be gained from using this integration:
- Support for capturing SSL-encrypted traffic
- Support for capturing Kafka traffic
- Blocking for Hive (also available using regular S-TAP), HDFS, and HBase
- Simplified deployment for auditing
- Redaction of result data is not supported
- S-TAP failover is not supported
This integration is available for Hortonworks 2.3 or later releases and Guardium 10.1 or later releases.
DB2 for z/OS enhancements
Guardium activity monitoring support for the z/OS® platform in V10 delivers increased scalability, performance, blocking and quarantining, and more options for collection of events. This section provides a high-level overview.
Performance and scalability improvements
- Multistream load balancing
As shown in the following graphic, multistream load balancing support enables the distribution of monitoring events in a round-robin fashion across up to six Guardium collectors. This helps to increase the scalability of the overall solution by offloading events quickly from z/OS and reducing the load on individual collectors.
Figure 62. Streaming events to multiple collectors
- The default for stage 1 filtering was changed to YES, which enables better performance without having to modify the configuration file.
- Improve separation of duties and reduce processor usage by removing reliance on IFI Trace for collection of DB2 commands, negative SQL Code events, SET CURRENT SQLID, and with a follow-on PTF, DB2 utility events.
- The collection of host variable data was turned off by default. This
improves security because host variables contain data, which you might
not wish to be exposed, and reduces processor usage by not requiring
the S-TAP to collect that information unnecessarily. You can enable
host variable collection on a rule-by-rule basis, as shown in this
sample DB2 Collection Profile policy rule.
Figure 63. Enable host variable collection on a rule-by-rule basis
More options for collecting security events
There are many enhancements that increase flexibility and improve the level of detail for auditing.
- Significant improvements in flexibility to collect events that are
related to negative SQL Codes. Similar to how it is specified on the
distributed platforms, administrators create groups of negative SQL
Codes in the Guardium appliance and specify that group on the DB2
Collection Profile policy rule as a white list or black list:
Figure 64. Creating negative SQL Code groups and specifying in a policy rule
You no longer need to enable IFI Audit class 1 to audit negative return codes, which might improve performance on the DB2 subsystem as well.
- Quarantine suspicious users to blocked access until investigation is
complete. Some organizations would like to enforce policies that
restrict privileged user access at certain hours and on certain
subsystems. Quarantining is set on a regular collector access rule
(not a DB2 collection profile rule), as shown in the
following screenshot. This quarantine rule applies only after the
events are received from the S-TAP.
Figure 65. Quarantine users policy
At runtime, S-TAP will check the database user against the quarantine policy on the Guardium collector. To avoid performance impacts to all other users, the SQL will still be processed. However, as soon as the verdict comes back from the Guardium collector, any subsequent activity by that user will be blocked. The figure below shows this flow.
Figure 66. Quarantining DB2 for z/OS users
- User issues SQL.
- If the SQL passes the collection profile filter, the event is streamed to the Guardium collector.
- If it is a match on the policy rule, the collector adds this user to a quarantine list and sends it to the S-TAP.
- The S-TAP stops further access from this user until the quarantine period is up.
Important: Because quarantining depends on data that passes the z/OS-side filtering, the z/OS collection profile must be a superset of the events that would match the quarantine policy. If this is not heeded, it is possible that S-TAP would never send an event to the collector that would match the policy rule, rendering the quarantine rule basically a no-op.
Quarantined connections appear in the Connections Quarantined report.
- Database name filtering on DB2 collection profile rules. You can now specify the database name (or group of database names), including wild carding, as an additional filtering option. To be effective, include the filter on all rules in the policy.
- Ability to capture CICS login user. Previously, the S-TAP would
capture the CICS user that was used to access DB2, which might be a
common ID. Now, the real login user is captured from the DB2 client
info fields for DB2 CICS applications.
Figure 67. CICS login user ID is captured in client info fields
- Supported for CICS Versions TS 4.2, TS 5.1, and TS 5.2.
- CICS login user ID reporting is not enabled by default. Enable
it by adding the CICS_USERID parameter to the ADHPARM file and
setting the parameter to Y. Example:
- In the CICS Connection definition, ATTACHSEC parameter must be set to ATTACHSEC(IDENTIFY) to pass the user ID from the Terminal-Owning Region (TOR) to the Application-Owning Region (AOR), which makes it available for collection.
In addition to supporting IMS V14, Guardium V10 offers the enhancements described here.
Reduce processor usage and improve performance
Here is a summary of performance improvements:
- Multi-streaming load balancing support. The support is nearly identical to that provided for DB2 for z/OS, described above in Performance and scalability improvements.
- Reduce overhead by optionally declaring BMP as "trusted" and disable
auditing. To disable BMP auditing, select the blue pencil in the Audit
field of the IMS Collection profile rule definition and select the
NOBMP check box as shown below.
Figure 68. Disabling the collection of IMS BMP events
- Reduce processor usage by limiting auditing to the lowest hierarchical
level of a DLI path call. This limits monitoring only to the last
segment of the hierarchical path instead of multiple segments when not
required. Currently, if you do a path call that touches 15 segments,
you would get 15 audit events. By disabling the higher level audits,
you can reduce that to one.
Disable higher level collection by clicking NOHLVL on the Audit field of the IMS Collection Profile policy in the same dialog box as shown in Figure 68.
Simplify installation and configuration
In V10, the S-TAP no longer requires a VSAM repository on the IMS host. All the definitions are now stored in the Guardium collector database.
- There is a new built-in report for IMS checkpoints, an example of
which is shown here:
Figure 69. Disabling the collection of IMS BMP events
- Before V10, only successful DLI calls were audited. In V10, you can
capture certain failure cases as well, which might help uncover
attempts to discover the system topology or otherwise gain knowledge
about the system. To add codes to the IMS collection profile rule,
click the blue pencil of the DLI Call Code field and
check the ones that you want audited, as shown in the following image.
Figure 70. Indicate which call codes to audit
- Support collection of application events that would not otherwise be
audited. There are some cases in which it is desirable to gather
non-database insights about such things as application or user events,
including, but not limited to:
- An alternate user name, such as those cases in which IMS Connect uses a generic name to connect to the database.
- A user-friendly application name.
- A change ticket number for whenever DBA changes need to be reconciled with a change ticketing system.
This collection does require some application and configuration changes that will signal the IMS S-TAP that a "special" event is being sent. The syntax and rules for the application event are consistent with other platforms and are documented in the Guardium Knowledge Center (see Related topics). To summarize, the first two bytes of the string is the ccsid of the encoding in hex format (only UTF-8 is supported). Following these first 2 bytes is a UTF-8 string in the following format.
Listing 1. String used to pass event information to the Guardium collector
SELECT 'GuardAppEvent:Start', 'GuardAppEventType:type', 'GuardAppEventUserName:name', 'GuardAppEventStrValue:string', 'GuardAppEventNumValue:number', 'GuardAppEventDateValue:date' FROM DUAL
Data sets for z/OS enhancements
Data sets auditing includes enhancements for load balancing, auditing enhancements, and setup configuration validation.
Load balancing provides the ability to offload data set events to multiple collectors and is nearly identical to that described for DB2 for z/OS.
Audit events can now be collected for data sets that reside on tape. No special configuration is required to enable this. An example report is shown below:
Figure 71. Events on tape data sets can be collected
Another thing you'll see on that report is the execute channel program (EXCP) counts for non-VSAM data sets, which basically tells you the number of reads and writes against these non-VSAM data sets. This allows for more granular reporting of access to partitioned data sets. Previously, you could not tell which specific member of a PDS or PDSE library was being accessed. In V10, the specific member name is reported.
Figure 72. The allocated member name for a PDSE is reported
You can now specify the DDNAME (as used in JCL) as a rule condition in the Data Set Collection Profile policy (see figure below).
Figure 73. DDNAME filter field in the Data Set Collection Profile policy rule
You can also add the DATA SET DDNAME as a column in the access report. Find it under the FULL SQL domain in the Query Builder.
Finally, a new setup check was added at agent startup to ensure that the proper SMF records are activated for auditing. The agent will continue to operate but without proper SMF setup, audit data cannot be collected. Here is an example that shows where SMF Records 17, 18, and 62 are not activated.
Figure 74. SMF test at agent startup notifies you if there is a problem
IBM i enhancements for enterprise-readiness
The S-TAP for IBM i was re-architected to support the following critical enterprise readiness features for scalability, high availability, and security:
- S-TAP load balancing to support:
- Failover of monitoring traffic to another collector if the primary collector is unavailable.
- Round-robin session data among a list of available collectors.
- Send duplicate data to multiple collectors for high availability.
- TLS encryption between the S-TAP and the Guardium collector.
As shown in the following figure, the overall architecture is largely unchanged from prior releases except that now most S-TAP configuration parameters are now stored in a guard_tap.ini file in the /usr/local/guardium directory on the IBM i server, similar to other database S-TAPs. Previously, they were stored in a table on the DB2 for the i database. Parameters that are related to filtering are still in the original table.
Figure 75. IBM i with Guardium architecture
Administrators configure the S-TAP using the same APIs and UI (S-TAP Control) as other UNIX S-TAPS. When the GUI or API is used to change the S-TAP configuration, the Guardium analysis engine (sniffer) sends a message to the S-TAP, which backs up the old .ini file, saves the configuration to the new .ini file and then restarts itself.
Administrators can set up encrypted communication and load balancing
options by using the S-TAP configuration controls and grdapi, such as
update_stap_config. Continue to use the
API for controlling
filtering parameters on the IBM i server.
Improved handling of Microsoft SQL Server encryption
Microsoft SQL server encrypts all of its logins by default. Optionally, all data can be encrypted by using the force-encryption option. Guardium must decrypt login (and optional) data traffic to be able to detect, for example, which database user is performing the activity. It did this on the appliance by correlating decrypted login information with incoming data streams.
In V10.1, Guardium significantly enhanced this capability. Now, decryption is occurring on the database client side. Client-side correlation eliminates all of the problems of appliance-side correlation by providing only decrypted network streams to the Guardium appliance. The appliance can inspect the contents of the traffic immediately, make decisions that are based on loaded policies, and then trigger actions based on those decisions.
This capability produces negligible processor usage on the server in initial performance testing in IBM lab. Your results might vary and I recommend that you test this in your own environment.
There are no configuration changes required, and it supports all Guardium-supported versions of SQL Server.
New S-TAP platforms
As always, see the V10 System Requirements document on ibm.com for the latest information. The following are new platforms supported by S-TAP:
- RHEL 7 x86_64
- RHEL 7.1 PPC64LE (PowerPC little endian, added in 10.1)
- SUSE 12 x86_64
- Ubuntu 14 (x86_64)
- Debian (supported via Ubuntu installer)
The following platforms are no longer supported:
- AIX 5.3
- SUSE 9
- Solaris 9
What's new in Vulnerability Assessment
IBM Security Guardium Vulnerability Assessment offers a comprehensive database vulnerability testing solution. In V9.5, the capabilities that were formerly only available in the Advanced Edition were made available to all current license holder for Vulnerability Assessment. Specifically, database entitlement reporting and the configuration audit system are now part of the single offering.
With V10, Guardium is including over 2000 tests. Guardium has a strong portfolio of tests that dig deep into hardening database security across many DBMS types. In many of these DBMS types, Guardium is either the first in the market or the only one who offers a solution.
New in V10 is several new database platforms, including DB2 for i, MongoDB, SAP HANA, Aster, new STIG Benchmarks for Oracle and SQL Server, and operational enhancements.
New database platforms
The following new database platforms can now be tested by using Guardium Vulnerability Assessment:
MongoDB is a popular NoSQL document-style database, which uses JSON-formatted documents for storing data. Guardium is the first and, as of this writing, only solution to support vulnerability testing for MongoDB. Supported releases include MongoDB 2.6 and 3.0.
Guardium includes over 50 tests that cover CVE patches, configuration best practices, and for users with elevated privileges. The figure below shows an improvement over time and an example of the kind of detailed recommendation you get with Guardium.
Figure 76. Test results include detailed remediation steps
Entitlement reporting and query builder are not supported for MongoDB.
DB2 for i
DB2 for i is a widely used database system across a variety of industries. Guardium has worked closely with the security subject matter experts in DB2 for i to come up with a comprehensive set of over 130 tests and entitlement reporting for IBM i 6.1, 7.1 and 7.2 partitions.
Types of tests include:
- Profiles with Special Authorities
- Profiles with access to Database Function Usage
- Password policies
- Database Objects privilege granted to PUBLIC
- Database Objects privilege granted to individual user
- Database Objects privilege granted with grant option
- Security APARs.
Entitlement reports include:
- Profiles with Special Authorities.
- Group granted to user
- Database Objects privilege granted to PUBLIC
- Database Executable Objects privileges granted to PUBLIC
- Database Objects privilege granted to individual user
- Database Objects privilege granted with grant option
Note: Configuration audit system is not supported for DB2 for i.
Ensure that you have proper prerequisites installed before running VA tests:
- IBM i 7.1 partitions:
- PTF Group SF99701 Level 26 or PTF Group SF99701 Level 25 with enabling PTFs SI50237, SI50251, SI50301 and SI51156.
- IBM i 6.1 partitions:
- PTF Group SF99601 Level 30
SAP HANA is an in-memory, column-oriented, relational database management system developed and marketed by SAP. Guardium is the first offering in this space for SAP HANA. Guardium has a suite of 65 tests. Types of tests include:
- Password policies
- Default SYSTEM password
- System privileges and roles
- Database Object privileges granted to PUBLIC
- Database Object privileges granted to individual user
- Database Object privileges granted with grant option
- Version and patches
- CAS (File permission and ownership)
Entitlement reporting is not currently available.
Teradata Aster is a database platform that is typically used for data warehousing and analytic applications. No other vendor offers vulnerability assessments for Aster.
Guardium supports Aster 5.1 and 6.1 and includes the following types of tests:
- Default password
- System privileges and roles
- Database Object privileges granted to PUBLIC
- Database Object privileges granted with grant option
- Version and patches
- CAS (File permission and ownership)
Create security assessments to run on the queen node as all database connections for Aster Data goes through the queen node only. Testing on worker and loader nodes is only required when performing CAS tests (file permission and file ownership).
Privilege tests loop through all the databases in a given Aster's instance.
DISA STIG Benchmarks for Oracle 11gr2 and SQL Server 2012
The Defense Information System Agency (DISA) publishes security technical implementation guides (STIGs), which are configurations that make existing applications or databases more secure. In Guardium Vulnerability Assessment Version 10, the external references for all existing and new STIG tests are in sync with the latest STIG benchmark.
- There are 34 new Oracle tests (authorization, privileges, and configuration) and five new SQL Server 2012 tests created from the latest STIG benchmark.
- The logic for many existing tests were modified to sync up with latest STIG recommendations.
- Tests that reference STIG now have a separate STIG reference and STIG severity fields. For Oracle, there is a also a STIG IA (Information Assurance) Control field (see Figure 77). For SQL Server, there is an SRG (Security Requirements Guide) reference.
- A STIG external reference for SQL Server now begins with SQL% instead of DG% or DM%.
Figure 77. Example STIG test for Oracle
The following configuration controls provide greater control over the behavior of VA tests:
- Configurable privilege test timeout value.
To avoid having entire test suites fail because of a problem with one test, there is now a query timeout configuration for both query-based and Java-based privilege tests. When a test takes more than 10 minutes to execute, it will time out with a message specific to the DBMS type driver. This mechanism can be turned off or modified by using the following CLI commands:
show va query_timeout
store va query_timeout off
store va query_timeout on <min>
A GUI restart is required.
Recommendation: Do not set the timeout value to greater than 30 minutes.
Restriction: Not supported for Aster, Informix, and SAP HANA.
- Configurable output limit for privilege tests.
To avoid the rare condition, which excessive violations cause memory issues, Guardium is limiting the number of rows returned per test to 20,000 rows. This default can be overridden by using the following CLI commands:
show va max_detail
store va max_detail off
store va max_detail on <num>where <num> is the number of rows
A GUI restart is required.
Restriction: Not supported for SAP HANA.
The purpose of this article was to give Guardium users a relatively detailed overview of the new features in the Guardium data security and protection portfolio. I hope that I accomplished that mission and that you will consider trying V10.1 yourself, which is really the best way for technical users to learn the product. Over time, we will continue to publish more articles, videos, and tech talks to help you get up to speed on this exciting new version.
- System requirements including database support for DAM.
- Guardium Knowledge Center
- The developerWorks article Accelerate the path to PCI compliance provides a good overview of the PCI accelerator. Although this was written for a previous release of Guardium, much of this material is still relevant even though it is now included in the base install for V10.
- The developerWorks article Use Guardium outlier detection to detect hidden threats provides a detailed overview of the outlier detection capability in Guardium, including configuration.
- Here is short blog by an IBMer on his experience with installing V10 for the first time.
- This Redbook publication entitled Deployment Guide for IBM InfoSphere Guardium is a must-read for anyone using Guardium. It is based on earlier releases, but many of the concepts are still relevant.
- This page documents the platforms supported by Guardium's File Activity Monitoring feature.
- This video demonstrates the Guardium File Activity Monitoring feature.
- This video gives you a guided tour of the Guardium V10 user interface.
- See live demonstrations of Version 10 by watching the replay of this Guardium Data Activity Monitoring Tech Talk.
- To capture IBM BigInsights HDFS traffic when GPFS is used, you will need to use the HDFS Transparency Connector.
- Two scenarios using the 10.1 investigation dashboard are shown in these video demos: possible misuse of shared user IDs and investigating failed logins.
- This video gives you a guided tour of the new Data Protection Dashboard in 10.1
- The developerWorks article Secure and protect Cassandra databases with IBM Security Guardium provides instructions for configuration and sample policy rules.
- See the replay of the Guardium 10.1 overview webcast.
- Watch the Guardium 10.1 tech talk.