Extracting data mart to table

Use data mart tables, for example, for frequently accessed data, systems with large-scale data that you want available for reports, and data you want to preserve after the scheduled purge. Data marts improve the performance of online reports on Guardium® aggregators.

Before you begin

Prerequisite: Data Mart Builder access rights (User Role).

About this task

There are no predefined data marts of the table type. You can create data mart tables from reports that have the data mart icon data mart icon. The data mart summarizes the data by hours, days, weeks, or months according to the granularity you specify. When you create a table data mart, Guardium creates a query-report with the name you assigned. You can modify this query-report just as you modify any other query-report. You can copy the query-report and modify it to suit your exact needs. Guardium also creates a custom domain and a custom table of the same name. External data correlation describes using custom domains and custom tables.

You can use the parameters in these reports to run functions (API) to generate scripts. See Working with API calls and reports.

Data Mart persistency: Changes to the original query or report do not affect the Data Mart; a snapshot of the originated analysis definition is saved together with the Data Mart upon creation.

When a data mart extraction runs (Scheduled or Run once now) for the first time, it extracts data from Initial start date to the current time based on the Time granularity. It saves the next period from in the DM_EXTRACTION_STATE table. On the next run, it extracts data starting from the next period from.

The extracted file name is <Global Id>_<short host name of source machine>_<file name defined by user>_<period start date time short format in UTC>.gz; for example: 1762144738_gibm32_LOGS_20181028230000_COMPLETE.gz

You can track the extraction and see the overall datamart status in the pre-defined Datamart Extraction Log report. View the User Defined Extraction Log for details on user-defined extractions, for example, datamarts and distributed reports.

To see a list of all data marts to which you have access, navigate to: Reports > Report Configuration Tools > Data Mart. From that page, you can open and modify the data marts.

After a data mart is created, you can modify its Purge After days, its archive schedule, and its schedule. To access the data mart, access the report from which you created the data mart, click Data mart icon, select the data mart from the list of data marts based on this report, and use the instructions in this procedure. If you have a data mart selected, and you want to create a new data mart, click New.

Procedure

  1. Access the report you want to create a data mart from and click Data mart icon.
    The Data Mart dialog opens.
  2. Enter a Data Mart name. Optionally, enter a description.
  3. In the Extract result to row, verify that Table is selected.
  4. Optionally, enter a Table Name. If not specified, it is saved as DM. It's helpful to define an intuitive name, since you can define indexes in the Custom Domain, and check table sizes using the GuardAPI.
  5. Specify a time granularity. This is the granularity of the resulting data mart table. Match your granularity to the frequency at which you'll run the corresponding reports: hourly, daily, monthly.
  6. The Archive/Export option controls whether the data from this data mart table is included in the data export and data archive, if these processes are configured on this unit. Select Yes if you want to enable data export and data archive for this data mart.
  7. Set the number of purge days. The purge days should reflect your business case, for example, the number of days you need that data for your reports, the size of data, available disk space.
  8. Select an initial start time from the calendar icon. This is the date/time from which you want the data extracted when the data mart extraction runs for the first time. For example, you define the data mart on Nov 5, 2018, but you need data from Nov 1, 2018. In this case, set Initial Start to Nov 1, 2018.
  9. Click Apply to save the Data Mart.
  10. To define data extraction on a regular basis, in the scheduling section click Modify Schedule, then define the data mart extraction schedule.
    • Start time: time of day the extraction starts
    • Restart: leave at Run only once
    • Schedule by: select Day/Week
    • Click Every day
    • Schedule Start Time: leave blank unless you want to start the datamart in the future. In that case, open the calenday and select the date on which to start exporting the data mart.
    • Automatically run dependent jobs. Leave unchecked. It is not relevant dor data mart.
  11. Select the user roles that have access to this datamart:
    1. Click Roles.
    2. In the Roles dialog, either select All Roles, or the individual roles.
    3. Click Apply. The roles are saved and the dialog closes.
  12. To temporarily stop report extraction click Pause, click Resume to resume report extraction.
  13. To run the extraction once in real time, click Run Once Now.