InfoSphere Guardium data security and protection for MongoDB, Part 3: Reporting, automating, and blocking

Protect and secure a new generation of databases

This article series describes how to monitor and protect MongoDB data using IBM InfoSphere® Guardium®, including the configuration of the solution, sample monitoring use cases, and additional capabilities such as quick search of audit data and building a compliance workflow using an audit process. Part 3 describes the capabilities in InfoSphere Guardium to search and report on audit data, how to create an audit process, and how to block users (available only with the Advanced Monitor). Many organizations are just getting started with MongoDB, and now is the time to build security into the environment to save time, prevent breaches, and avoid compliance violations.

Share:

Indrani Ghatare (indrani@us.ibm.com), Software Engineer, InfoSphere Guardium, IBM

Indrani GhatareIndrani Ghatare has been a software developer at IBM for more than 12 years. Currently, Indrani is a member of the Research and Development team for InfoSphere Guardium Collector. Indrani has worked on the development of the MongoDB parser and the logger component of Guardium Collector.



Matt Kalan (matt.kalan@10gen.com), Senior Solutions Architect, 10gen

Photo of Matt KalanMatt Kalan is a senior solutions architect at 10gen, based in New York, with extensive experience helping more than 200 customers in financial services and other industries solve business problems with technology. Before 10gen, Matt grew Progress Software's Apama Algorithmic Trading and Complex Event Processing (CEP) Platform business in North America and later sold broader operational intelligence solutions to financial services firms. He previously worked for Caplin Systems selling solutions to stream real-time market data over the web to FX and FI portals, and for Sapient providing consulting services to global 2000 clients.



Sundari Voruganti, QA Specialist, InfoSphere Guardium, IBM

Sundari Voruganti photoSundari Voruganti is a member of the InfoSphere Guardium QA team at IBM Silicon Valley Lab. Sundari has been with IBM over a decade and has a diverse background in both engineering as well as customer enablement roles. As a passionate technologist, she loves the challenge of learning and working with new technologies as well as helping customers understand and implement IM solutions. Sundari has a double Masters in Computer Science from Bangalore University and University of Alberta.



Kathryn Zeidenstein (krzeide@us.ibm.com), InfoSphere Guardium Evangelist, IBM

Photo of Kathryn ZeidensteinKathy Zeidenstein has worked at IBM for a bazillion years. Currently, she is working as a technology evangelist for InfoSphere Guardium data activity monitoring, based out of the Silicon Valley Lab. Previously, she was an Information Development Manager for InfoSphere Optim data lifecycle tools. She has had roles in technical enablement, product management and product marketing within the Information Management and ECM organizations at IBM.



27 June 2013

Also available in Chinese

Introduction

The first part of this article series introduced MongoDB and InfoSphere Guardium. It provided some background on how InfoSphere Guardium can help you feel more secure about using MongoDB in more applications because of its ability to thoroughly monitor and audit data activity. You learned the architecture of the solution and some of the recommendations on securing MongoDB using its native capabilities as well as the benefits that InfoSphere Guardium brings.

The second part of the series got into the nitty gritty details of configuring the solution and explained how to create security policy rules for a variety of use cases.

This final article in the series describes more of what you can do with InfoSphere Guardium, including:

  • Searching and browsing audit data.
  • Creating reports, and configuring an audit process to route the report for review.
  • Blocking suspicious users. This particular capability is only available with the Advanced Activity Monitor.

Viewing audit results: Searching and reporting

This section describes a couple of different options for working with audit data:

  1. Ad hoc search (new in this release)
  2. Activity reports

Search and browse

The quick search facility, available from the top banner of the Guardium UI when enabled, provides fast search and browse capability for database activities, errors, and violations. This capability uses search indexing, which runs every one to two minutes. You can use common search techniques, including faceted search and text search, together or in combination to quickly do ad hoc investigations of audit data.

The search includes the following:

  • A histogram view of statistics of the who, where, what, and when facets of database activities. Click on any of these facets to narrow your search.
  • Free-text search that searches all indexed audit data. For example, you could enter the string 'john' and it will search all indexed data to find john, whether it appears in the DB User, OS User, Client, Error, in the message details, and so on.

The search and browse feature is disabled by default. To enable it you can run the following grdapi as a CLI user: grdapi enable_quick_search.

To access this capability, enter a search term or click on the flashlight in the top banner of the InfoSphere Guardium UI, as shown in Figure 1.

Figure 1. Accessing search and browse of audit data
Flashlight icon highlighted in a red box

Use the facets on the left to choose value to add to the filter. For example, if you want to see if anyone has recently dropped an object, you would choose your timeframe and then select MongoDB as the DB Type and drop as the verb.

Figure 2. Browsing through audit data using the histogram (facets)
Browsing through audit data using the histogram (facets)

This narrows the result and includes details, including the object dropped, the time, and the db user who issued the command. Due to space constraints, not everything is shown in Figure 3.

Figure 3. Filtered search results
Filtered search results

Creating reports

InfoSphere Guardium includes many built-in report definitions as well as a rich report building capability. Some reports are populated with data automatically; for others, you must modify the run time parameters to populate the report, such as specifying the correct date range.

If you see a report you like, you can also clone the query that is used to generate that report and tweak it to suit your needs. For example, let's assume you like the report called "Admin Users Login." If you are logged in as an admin, you will find this report on the Daily Monitor tab. If you click on the report, you may see the message No data found, as shown in Figure 4. The usual causes for this message are if the date range does not include a timeframe where that activity appears or if some other aspect of the query condition is not met.

Figure 4. A built-in report in which query results are not found
A built-in report in which query results are not found

By clicking on the yellow pencil below the report, you can see the base query used to create the report, which looks like Figure 5. The top half includes the columns in the report and the bottom includes the conditions. You can see that one condition for the report is a runtime parameter for the Admin Users group.

Figure 5. The query used to generate the report
query builder shows report columns and report conditions

To see who is in the Admin Users group, you can go to the Group Builder tool. Assuming you are logged in as an admin user, navigate to the Group Builder from the Tools tab.

Figure 6. Group Builder menu
shows Tools tab and Group Builder menu item highlighted

When the Group Filter opens, click Next. From the list of existing groups, select Admin Users, and click Modify.

If your MongoDB admin users are not in the list, you can add them, as shown in Figure 7.

Figure 7. Adding a new administrative user to the group
shows adding sundari as an admin member

If the report still doesn't show data, you can customize the report to change the data range. To do this, click on the Customize icon (blue pencil ) in the upper right of the report, as shown in Figure 8.

Figure 8. Customize the report by clicking on the blue pencil in upper right of report portlet
text in figcap is descriptive

Then you can change the date range (as we have in Figure 9) to include a time range that includes the entire previous week (NOW -1 week), when we know there was activity.

Figure 9. Extending the date range of the report
date changed to NOW-1 WEEK to inclue more data

Now the report shows data, such as shown in Figure 10.

Figure 10. Report now shows data
Report shows activity of SUNDARI logging in

We've barely scratched the surface of report customization and creation. However, in Resources there is a link to a "how to" topic in the InfoSphere Guardium Information Center that can help you get started.

Use a data mart to improve performance of predefined reports

You can use your own or Guardium predefined queries and reports to create a data mart that you can use to significantly improve the performance for your frequently used reports. A data mart is a designated table that is based on your desired report structure. You define the data mart's data granularity for summarization purposes (such as minutes, hours, days, or weeks) and schedule it to run periodically. Because the structure is predefined, it eliminates the need to do runtime joins or other resource-intensive database activities.

Data marts can be especially effective when used with the regular reporting that you may need to do on the Guardium Aggregators on large amounts of data. Because the data mart reflects the structure of the report, there is no need for complex, multi-table joins, thereby significantly improving the performance of those reports.

Any report can be turned into a data mart by clicking on the Data Mart Builder link at the bottom of the report, as shown in Figure 11.

Figure 11. Icon to go into the Data Mart Builder
a report highlighting the location of the icon in the lower right part of the report (bottom toolbar)

In the Data Mart builder, you can specify how often you want the query to run, and when you save it a default report is created using that definition. The data is populated continuously per the granularity and schedule that you select.

Figure 12. Data Mart Configuration
data mart config fields include data mart name, description, extract result to table, table name, time granularity, etc. click on Add to Pane to add the corresponding report to the UI.

Creating a report of audit data and adding it to a workflow for review

Running reports and getting approvals is not anyone's idea of fun. The more this process can be automated, the less time you have to spend worrying about it and the greater your ability to have the audit trail you need for compliance officers or auditors. With Guardium, you can use an audit process to schedule the reports to run and be sent automatically to the reviewers. The system will keep track of who has reviewed or signed off. All required actions can be securely executed through the web interface, including reviewing results, providing approvals, commenting, and escalating an action.

Figure 13. Audit process can automate routing of reviews and signoffs
Arrow shows flow through DBA to Information Security to Auditor/Manager

Although there's not space to go into great detail about creating an audit process, we did want to show you the highlights. In this example, we want to create an audit process that sends a copy of our Privileges report to the appropriate person for review.

Figure 14 shows what our report looks like.

Figure 14. A report we created called MongoDB Privilege Report
A report that includes activity on system.users

To add it to an audit process, go to the Audit Process Builder (from Tools, if you are logged in as admin) and create a new process. You will be asked to create receivers, who are the people (or roles) who will receive the results of the audit process. Then you add the report as a task in this audit process, including the time period you want the report query to cover (for example, NOW -1 DAY until NOW). A process can have multiple tasks, which might consist of a set of reports or other activities.

Figure 15. Audit process builder
highlights the areas of the builder such as the schedule, receivers, and the Audit Tasks portion of screen, which is where you can add the mongodb privilege report review task

You can schedule the task to run periodically, such as every day at midnight. That way, administrators or DBAs will have the report in their email in the morning. Their approvals (if required) and comments will be kept as part of the audit trail for this process.

Figure 16 gives an example of an email that was sent that includes a PDF of the report.

Figure 16. Example e-mail from audit process
emails says for your review: users and privileges on MongoDB Audit process

Blocking access (with Advanced Activity Monitoring)

There may be cases in which it is critical to block users from access to particular data. A typical use case would be blocking administrator access to sensitive data or when an unknown user is discovered in the system. This capability, which is available in the Advanced Activity monitoring license, can help protect your data when:

  • Outsourced DBAs are used who need to be restricted from access to sensitive data
  • When there are concerns over an existing or imminent insider or external breach

This capability relies on S-GATE, which is an extension of the S-TAP that you activate by modifying some parameters in the guardtap.ini file. The S-GATE parameters are called "firewall" parameters. Because the use of S-GATE can introduce some latency, we recommend that you enable S-GATE in the initialization file but that you turn it on only when certain specific conditions are met using a policy action called S-GATE ATTACH. Listing 1 provides an example of the firewall parameters you modify in the guardtap.ini file.

firewall_installed=1    <This enables the S-GATE capability to be used
firewall_fail_close=0
firewall_default_state=0  <S-GATE off by default but can be turned on using a policy  rule
firewall_timeout=10

Assuming that the S-GATE ATTACH rule has been activated, the process works as follows (See Figure 17 for an illustration of the steps):

  1. When an unauthorized user connects to MongoDB and enters a command, the S-GATE recognizes that this particular session is one that must be watched.
  2. The command is held by S-GATE.
  3. The command is compared against the policy to see if there is a policy violation, such as accessing a particular sensitive object or group of objects.
  4. If the policy rule is violated and your policy action is S-GATE TERMINATE, S-GATE will terminate the session and the command will not be allowed to continue to the Mongo server. If the command is allowed, then the statement is allowed to go to the server.
Figure 17. Overview of the blocking process
Overview of the blocking process

There are many different combinations that can be used to set up blocking for your needs. The most important thing is to ensure that you do not use this for normal business applications because of the latency introduced to check all the incoming commands against the policy.

For our policy, we will block access to any sensitive data objects from anyone who is not in the "authorized user group." This group may consist of specific IDs for an authorized application, for example. Table 1 is a summary of the rules we'll be creating and the groups that are used in each rule.

Table 1. Rules and groups used to create our sample policy rules
Rule descriptionGroups used in the rule
Watch unauthorized users (Enable S-GATE for any activity by a user who is NOT in the authorized user group for these objects.) Authorized users (modify built-in group)
Block unauthorized users (to sensitive data) Authorized users (modify built-in group)
MongoDB Sensitive Objects (created earlier, in Part 2 of this article series)

Now we'll describe the rules you need to activate blocking under the right circumstances.

First, as shown in Figure 18, specify the S-GATE attach conditions in which everyone who is not an authorized user will activate S-GATE. Note that this will have a performance impact, so you only want to activate S-GATE attach under limited circumstances. It is not recommended to use this with application traffic.

Figure 18. Set the conditions under which to start watching data activity
The condition is NOT authorized users, and the action is S-GATE Attach

Next, if any of those users access sensitive data, go ahead and terminate the connection.

Figure 19. Set the conditions under which to terminate the user's connection
added mongodb sensitive objects group to condition and action is S-GATE Terminate

Notice that when a session is blocked, this will get written as a policy violation, which you can also see on the Incident Management tab. If your rule includes an action to alert, this will also get written to SYSLOG so that it can be forwarded to a SIEM system for follow-up and investigation by security personnel. There is also an option to quarantine a user for a specified period of time to allow for security investigations to take place.


Summary

This article series has done a deep dive on the InfoSphere Guardium support for MongoDB, but the capabilities described herein are applicable across a wide variety of database systems, which reflects the widely heterogeneous data processing systems that represent both past, present, and future database technologies. Database and data processing technologies evolve rapidly to fulfill the needs of new applications and use cases. MongoDB is one of the leading providers of "NoSQL" technology, and as such, is being used alongside traditional relational databases in many organizations who need new technologies to handle new data processing problems. Thus, having a data security and monitoring solution that can also be used across both relational and NoSQL technologies just makes sense.

InfoSphere Guardium is built on a foundation that can adapt and quickly deliver data monitoring capabilities across a wide variety of databases, including other NoSQL systems such as Cassandra and CouchDB, relational, mainframe data (including DB2, IMS, and VSAM) or Hadoop-based systems such as InfoSphere BigInsights, Cloudera, Greenplum Hadoop, and Hortonworks Data Platform.

In this article series, we have demonstrated the result of work that IBM and 10gen have done together to develop and validate InfoSphere Guardium capability for MongoDB. Together, these offerings can provide a NoSQL option that offers the security and auditing capabilities often required by organizations for compliance with a wide variety of privacy and financial regulations.

Resources

Learn

Get products and technologies

Discuss

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Information management on developerWorks


static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Information Management, Security
ArticleID=935043
ArticleTitle=InfoSphere Guardium data security and protection for MongoDB, Part 3: Reporting, automating, and blocking
publish-date=06272013