Monitoring a Data Gate instance

The dashboard is displayed after the creation of an instance has finished. You can also access the dashboard from the Cloud Pak for Data main menu. The dashboard gives you various pieces of information about your Data Gate instance, such as the system availability, the table status, the replication latency, and throughput. The dashboard also allows you to add, load, and remove tables or to enable or disable synchronization. The dashboard is refreshed every minute.

Overview tab:

• Viewing the status of the instance on the Instance: <name> tile:
  The tile contains the following blocks of information:

  Instance name

  The instance name functions as the header of the information box.

  Target

  Under the Data Gate instance name, you find the instance name of the selected target database (Db2 or Db2 Warehouse). In the background, a test SELECT statement is run to verify that the database is available. If the test statement can be run successfully, you see a green check mark.

  Source

  Under the target instance name, you find the Db2 location name followed by the @-sign, the hostname or IP address of the connected data source (Db2 for z/OS subsystem or data sharing group), and the network port that is used to connect to the source. In the background, a test SELECT statement is run to verify that the Db2 for z/OS data source is available. If the test statement can be run successfully, you see a green check mark.

  Data Gate server

  The readiness and availability of your instance. If it is in working order, you see a green check mark followed by Data Gate server active.

  Synchronization

  The state of the synchronization function. If it is in working order, you see a green check mark followed by the message Synchronization started. You can click the toggle button on the right to start or stop synchronization for your instance.

  Query acceleration

  If you selected Use this Data Gate instance for query acceleration as you create the instance, you also see the query acceleration status. If query acceleration is in working order, you see a green check mark followed by the message Query acceleration started. You can click the toggle button on the right to start or stop query acceleration for your instance.

• Viewing the status of Data Gate table metadata in IBM Knowledge Catalog on the IBM Knowledge Catalog integration tile:
  This tile contains the following information:

  Status

  The status of IBM Knowledge Catalog integration. Possible status values are:

  Not configured

  The IBM Knowledge Catalog service is not configured. Make sure you completed all steps in Configuring Data Gate table metadata publishing to IBM Knowledge Catalog successfully.

  Available

  The IBM Knowledge Catalog service was integrated successfully. The service can be reached from the Data Gate instance.

  Error

  The IBM Knowledge Catalog service is installed, but cannot be reached by Data Gate. If the status is Error, run the following checks:

  1. Check if the IBM Knowledge Catalog service is up and running.
  2. Check if all Data Gate pods are up and running.

  Last published

  A timestamp or the information N/A. A timestamp indicates the last time metadata was published to IBM Knowledge Catalog. If the information N/A is displayed, it means that metadata has not been published until now.

  Publish to catalog

  If the Status is Available, you can click this button to publish Data Gate table metadata to IBM Knowledge Catalog.

  For more information on IBM Knowledge Catalog integration, see Publishing Data Gate table metadata to IBM Knowledge Catalog.

• Viewing the status of tables or queries on the Table or Query tile:
  The Table tile contains the following information:

  The pie chart on the left of the tile shows the distribution of table statuses among the tables that were added to the Data Gate instance. The bigger a colored segment, the greater is the number of tables in the state that is indicated by the color.

  A text box on the right of the tile starts with the total number of tables managed by the Data Gate instance. What follows is the legend of the color scheme used in the pie chart. Next to each colored bullet, you find the number of tables in that particular state:

  Green: Active

  The number of tables that are loaded and take part in the synchronization process.

  Purple: Loaded

  The number of tables that are fully loaded, but do not take part in the synchronization process.

  Blue: Loading

  The number of tables that are currently being loaded.

  Yellow: Load pending

  The number of tables that still need to be loaded.

  Red: Error

  The number of tables in Error state.

  The Query tile shows information about the queries that were routed to your Data Gate instance. If Use this Data Gate instance for query acceleration has been selected during the creation of the instance, you see a pie chart with a color scheme similar to that on the Table tile. The color scheme works in the same way. In addition, the tile gives you a few statistics on the query execution by your Data Gate instance:

  Green: Successful

  The number of queries that were completed successfully.

  Purple: Executing

  The number of queries that are currently being executed.

  Blue: Queued

  The number of queries waiting to be executed.

  Red: Failed

  The number of queries that could not be completed successfully, but had to be aborted.

  Note: The query numbers on this tile are always counted from the last restart of the Data Gate server, Data Gate server pod, or query accelerator. The values numbers are reset to zero when one of these components restarts, and are then newly calculated from the time of the restart. They can therefore deviate from the number of listed queries on the Query table. That is, the numbers displayed on this tile might be lower than the number of the counted queries on the Query tab. The Query tab lists all queries that were routed to a particular Data Gate instance, including the time before the last restart of one of the mentioned components.

Monitor tab:

• The graphs on the Monitor tab show the synchronization latency, synchronization throughput, the target data, and a few statistics regarding the source data and the logs. The graphs are synchronized, meaning that they all cover the same time frame on the x-axis. If you click a point on the x-axis of a graph, the selected point in time is displayed under Time range on the right side of the tab window. Underneath, you see the selected value and the corresponding values in the other graphs. For the Synchronization latency and the Synchronization throughput, the average values (average of the values plotted on the y-axis) are also shown on the right side of the tab window. If you hover over a data point in a graph, the Time range display changes to Time point, and the date and time related to this point are shown underneath.

  Using the slider buttons on the left and right sides of the bar at the bottom (under the last graph), you can narrow the focus (time frame) of the display. The left slider button changes the beginning of the time frame on the display. The right slider button changes the end of the time frame. Shorter time intervals allow you to scrutinize individual values more closely. Wider time intervals offer a broader perspective.

• You can change the time frame that is covered by the graphs on the Monitor tab. The labels on the buttons indicate how far the data that is taken into account reaches back into the past.
  • 1 hour
  • 6 hours
  • 12 hours
  • 24 hours
  • 3 days
  • Custom. This button allows you to specify a time frame of your choice.
• Export button:
  The Export button allows you to save the collected monitoring data to a file in comma-separated variable (csv) format. You can import this file into other tools, such as a spreadsheet, for reporting purposes or further analysis. The name of the export file downloaded to your local machine is monitor_data_<start_time>_to_<end_time>.csv, where <start_time> and <end_time> denote the beginning and the end of the time frame covered by the export file. The start time and end time of the file match the time frame that you selected on the Monitor tab (see previous bullet).
• Viewing the synchronization latency:
  The Synchronization latency graph illustrates the development of the synchronization latency over time. At any given moment in time, the replication latency tells you how long it takes to bring your Data Gate tables into a state that matches the state of the original Db2 for z/OS tables, that is, a state in which no more updates are necessary. In other words, it is the time that your Data Gate tables are lagging behind with regard to synchronization. Ideally, this should only be a few seconds.
• Viewing the synchronization throughput:
  The Synchronization throughput graph can be seen as a performance curve of the synchronization function. It shows how many updated table rows per second where propagated to your Data Gate tables.
• Viewing source data and log statistics:
  The Source data graph tells you how many operations of a certain type were carried out on the source database during the synchronization process, how many compensated rows exist, and how many log records were written by Db2 for z/OS utilities. By default, compensated rows and log records are deselected, but you can click the corresponding entries on the legend to enable these. The statistics in detail:

  Source inserted rows

  A curve that shows the number of synchronized table rows that were inserted into the source database.

  Source updated rows

  A curve that shows the number of synchronized table rows that were updated in the source database.

  Source deleted rows

  A curve that shows the number of synchronized table rows that were deleted from the source database.

  Source compensated rows

  A curve that shows the number of synchronized table rows (INSERTs, UPDATEs, DELETEs) in the source database that were ignored because the operations were not committed, but rolled back.

  Utility logs

  A curve that shows the number of log records that were generated by Db2 for z/OS utilities.

• Viewing target data statistics
  The Target data graph shows you the number of operations of a certain type that were carried out on the target database during the synchronization process, and the number of net-effect operations. By default, net-effect operations are deselected, but you can click the corresponding entry on the legend to enable these. The statistics in detail:

  Target inserted rows

  A curve that shows the number of synchronized table rows that were inserted into the target database.

  Target deleted rows

  A curve that shows the number of synchronized table rows that were deleted from the target database.

  Net-effect operations

  A curve that shows the number of net-effect operations on the target database. This is the number of unnecessary operations that the database engine skips to optimize the performance.

Table tab:

• Viewing the tables of the instance:
  The Table tab gives you detailed information about the tables you selected for the instance:

  Table	Schema	Table size	Sync
  Table name	Name of the schema a table belongs to	Table size in MB, GB, or TB	Whether synchronization is enabled or disabled for a table

  Sync started	Status	Acceleration	Archived rows	Last statistics collection
  Timestamp showing when the synchronization process was started for a table	The status of a table (for a legend, see the Tables section in the instance status box above)	Whether the query acceleration feature is enabled or disabled for a table	The number of table rows that have been archived	Timestamp that shows the last time the RUNSTATS utility was used to collect statistics on a target table

  Note: The column Acceleration is shown only if Use this Data Gate instance for query acceleration has been selected during the creation of the instance.

  Similarly, the Archived rows column is shown only if the archiving function was enabled.

• Searching for particular schemas or tables:
  To display just the schemas and tables whose names contain a particular search string, type a search string in the Find field.
• Setting a filter:
  To display just the tables in a particular state or to limit the number of tables in the list, click

  Click Apply filters when finished. To remove the filter, click the icon again and click Reset filters

• Downloading the tables list:
  To download the tables list in csv format, click .
• Importing the tables list:
  To import the tables list in csv format, click .
• Refreshing the tables list:
  To refresh the information in the tables list, click .
• Adding tables:
  To add tables to the instance, click the Add table button on the header of the tables list. For more information, see Adding tables to Data Gate.
• Enabling or disabling synchronization for tables that were added to the Data Gate instance:
  To enable or disable table synchronization, select one or more tables (check boxes in front of table names) and click one of the following on the header of the tables list:
  • Enable
  • Disable
  For more information, see Enabling or disabling table synchronization.
• Loading tables:
  To load tables, select one or more tables (check boxes in front of table names) and click the Load button on the header of the tables list. For more information, see Loading tables in Data Gate.
• Removing tables:
  To remove tables from the instance, select one or more tables (check boxes in front of table names) and click the Delete button on the header of the tables list. For more information, see Removing tables from Data Gate.
• Reducing or increasing the number of items per page:
  To reduce or increase the number of table rows displayed per page, click Items per page. You can select between 8 (default), 20, or 50 table rows per page.
• Selecting a start page:
  To select a particular start page for the tables list, click the drop-down icon next to the page number at the bottom of the tables list and select a different page number from the selection list that opens. For example, to show page 100 as the start page of the tables list, click the drop-down icon and select 100.
• Navigating between the pages of the tables list:
  To change back and forth between the previous or next page of the tables list, click the < or > button at the bottom of the list.
• Enabling or disabling query acceleration:
  If you selected Use this Data Gate instance for query acceleration during the creation of the instance, you can enable or disable query acceleration for individual tables. To do so, select one or more tables (check boxes in the front of table names) and, depending on the current acceleration state of the table, click one of the following buttons on the header of the tables list:

  • Enable acceleration
  • Disable acceleration
• Archiving table partitions:
  Select the tables (check boxes in front of table names) that you want to archive from. Then click Archive on the header of the tables list to continue. For more information, see:

  • Data Gate storage saver
  • Archiving data from the Data Gate dashboard
• Restoring table partitions:
  Select tables (check boxes in front of table names) that contain archived partitions you want to restore. Then click Restore on the header of the tables list to continue. For more information, see:

  • Data Gate storage saver
  • Restoring data archived by Data Gate

Query tab:

The Query tab is available only if you selected Use this Data Gate instance for query routing during the creation of the instance.

• Viewing queries that were routed to the instance:
  The table on the Query tab gives you detailed information about the queries that were routed to the instance (accelerated queries):

  Query	Username	State	Start time	Execution time
  The SQL statements of submitted queries.	The names of the users who submitted past queries.	Processing states that the queries are in.	Times when queries were received by the Data Gate instance.	For each query: the time that the query "spent" in the queue before it was routed to the Data Gate instance plus the time the instance needed to process the query.

  Elapsed time	Wait time	Result size	Rows returned
  The total times needed to process the queries, beginning with the state QUEUED and ending with either FAILED or SUCCESSFUL	The times that queries had to "spend" in the queue before they were processed by the Data Gate instance.	The data sizes (in KB, MB or GB) of the query results that were returned by the Data Gate instance to Db2 for z/OS.	The numbers of rows that were returned in query results.

  Note: The Query tab lists all queries that were routed to a particular Data Gate instance, including the time before the last restart of the Data Gate server, Data Gate server pod, or query accelerator. That is, the number of listed query can deviate from the number that is displayed on the Query tile of the Overview tab. In most cases, the number of listed queries is higher than the number displayed on the tile. This is because the values on the tile are reset to zero when the server, server pod, or accelerator restarts, and are then newly calculated from the time of the restart.
• Searching for particular queries:
  To display just the queries that have a particular search string in the SQL statement, type the search string in the Find field.
• Setting a filter:
  To select the columns to be displayed in the table on the Query tab, click

  A sub-window opens. To show all available columns, click and select the check box in front of Show all columns. Then click OK.

  To remove columns from the display, clear the check boxes in front of one or more column names, and click OK.

• Viewing the full SQL statements:
  If the SQL statements in the Query column are too long to be displayed in full, they are truncated. To view the full SQL statement of a query none the less, select the query in the table (check box in front of the table row) and click Full query on the header of the table.
• Downloading list of the accelerated queries:
  You can download a list of the queries on the Query tab in csv format.
  To do so, click on the header of the table.
• Refreshing the table of queries:
  To refresh the information in the queries table, click on the header of the table.

The Event tab:

The Event tab lists synchronization event messages. The tab lists recent messages and offers filters that you can use to narrow the number of messages on the display.

• Viewing synchronization event messages:
  The table on the Event tab lists the available messages.

  Severity	Message	Event ID	Originator	Time
  The severity classification of a message, that is, one of the following: Information Warning Error	The message number and the message text.	A number used for the classification of an event.	The name of the program component that caused an event.	A timestamp that shows when an event occurred.

  For a complete list and descriptions of all messages that might be displayed, see DSNX881I Messages (ID 20) Returned by IBM Integrated Synchronization. The synchronization function of Data Gate returns exactly the same messages.

• You can change the time frame that is covered by the events table. The labels on the buttons indicate how far the data that is taken into account reaches back into the past.
  • 1 hour
  • 24 hours
  • 7 days
  • 30 days
  • Custom. This button allows you to specify a time frame of your choice.
• You can filter the synchronization messages by their severity. From the Event severity drop-down list, select one or more severity classifications:
  • Information
  • Warning
  • Error

  Selecting one or more of these classification types lists the messages classified in that way. If, for example, you select Warning, you will see warning messages only.

• You can export and download the list of messages on display in comma-separated value (csv) format. To this end, click and specify a file location on your local computer.