Product Documentation
Abstract
Best practices checklist for deploying ISIQ
Content
This document recommends a process for deploying IBM Security Verify Information Queue (known by the acronym “ISIQ”) in a new docker swarm for the purpose of integrating Governance and Identity Manager data. The process is divided into three phases, with an estimated time for each. The phased approach is a best practice for achieving successful implementations.
The assumption here is that someone relatively new to docker and ISIQ is doing the work. If it’s an experienced practitioner, shrink the time estimates. As a short-hand, the product acronyms "IGI" and "ISIM" are used throughout this document, but these instructions are also applicable to the v10.0.x levels of IBM Security Verify Governance and Identity Manager.
|
Action |
References/Comments |
|
Read about docker swarm, follow the tutorial |
https://docs.docker.com/engine/swarm/ and https://docs.docker.com/engine/swarm/swarm-tutorial/ |
|
Review the ISIQ documentation set |
|
|
Provision a Linux VM for a test ISIQ in a single-node docker swarm configuration |
See “System Requirements” in the Deployment Guide. Setting up a single-node configuration is intended to give you simple, hands-on experience with docker and ISIQ. |
|
Install latest Docker CE (v23.0 as of Feb. 2023) on the provisioned Linux VM |
See the installation information at https://docs.docker.com/install/ The instructions in this document assume VMs are used to run ISIQ, but physical Linux systems would work as well. |
|
Go to the ISIQ starter kit page and download the starter kit .zip file to the Linux VM; extract the contents of the .zip file |
https://www.ibm.com/support/pages/ibm-security-information-queue-starter-kit - After you extract the .zip, you will find an “isiq” shell script in the root folder. The script simplifies common ISIQ deployment functions. For details, refer to “Using the isiq shell script” in the Deployment Guide. |
|
Pull the required images, and create an SSL certificate for nginx by running: <starterKitDir>/isiq setup |
`isiq setup` pulls ISIQ’s docker images and performs security configuration. This step and subsequent steps are described in “Summary of Installation & Configuration Steps” in the Deployment Guide. Note: You need approximately 8 - 10 Gigabytes of free space on the node where you pull all of the docker images. |
|
Configure OIDC by updating <starterKitDir>/cfg/oidc/oidcSettings.json |
If you don't have a preferred OpenID provider, you can specify IGI as the provider for ISIQ. Refer to the “OpenID” section of the Deployment Guide. If OIDC setup will take a while to complete, you can temporarily bypass OIDC by enabling the ISIQ_SKIP_OIDC environment variable. For more information, refer to “Shortcuts for Rapid ISIQ Setup“ in the Deployment Guide. |
|
Review the default <starterKitDir>/cfg/connect/txdef.json file for possible modification |
This is an important data analysis step and will vary in scope depending on whether, and to what extent, you have customized ISIM and IGI. The contents and format of txdef.json, and how to modify it, are discussed in the “Custom Transformations” section of the User's Guide. |
|
Create the docker swarm |
See Step 8 in the “Summary of Installation & Configuration Steps”. |
|
Review the <starterKitDir>/yml/single_node files that deploy a single-node docker swarm |
The three required .yml files are broker-stack, connect-stack, and app-stack. The `isiq start` script initializes them in that sequence. logs-stack.yml is optional (and can be implemented later) for self-monitoring. You shouldn’t need to edit the single-node .yml files. The defaults are usually adequate. |
|
Start the ISIQ docker stacks |
See “Deploying the ISIQ stacks” in the Deployment Guide. |
|
Verify the stacks started and the OIDC provider lets you log in |
“Verifying the stacks are deployed” in the Deployment Guide and, if need be, “Diagnosing Service Startup Failures” in the Troubleshooting Guide. |
|
(Optional) Configure the logs stack with Elasticsearch, Logstash, Kibana, and Grafana |
ISIQ’s logging & monitoring example uses ELK (Elasticsearch, Logstash, Kibana) technology and is described in the “logs-stack.yml” section of Appendix A in the Deployment Guide. If you have logging & monitoring tools you prefer to use instead, you can do so. These particular components aren't required, nor are you required to run a logs stack. In general, a logs stack offers more value in a cluster than in a single-node configuration. |
|
Configure your test ISIM as an ISIQ product |
For instructions, refer to “Configuring ISIM” in the User's Guide. |
|
Verify that all ISIM entities are being read and stored in ISIQ’s directory topics |
Various issues can arise when reading ISIM’s LDAP directory. Therefore, before proceeding further we recommend you validate that all expected LDAP entities are being stored in ISIQ’s “directory” topics. It’s easier to debug these issues if you have a simple ISIQ environment with only one ISIM product configured. See “Verifying ISIM Data Loading” in the User's Guide. |
|
Configure your test IGI as an ISIQ product |
For instructions, refer to “Configuring IGI” in the User's Guide. |
|
Apply IGI customizations |
Several IGI rule flow updates and other customizations must be made before you start integrating ISIM data. Refer to “Appendix A: IGI Customizations for ISIQ” in the User’s Guide. |
|
Subscribe the test IGI to the test ISIM |
In the ISIQ UI, navigate to the dashboard of your test IGI product and click the “Subscriptions” dropdown to define a subscription. This action initiates ISIM-to-IGI data integration. For more information, refer to the "Subscriptions" section of the User's Guide. |
|
Log in to your test IGI and navigate to the "Monitor" tab of AGC to observe how test ISIM entities are flowing across to IGI |
At first, the Users, Accounts, Organizations, etc. from ISIM should be listed as "Unprocessed" events in IGI. After a while, they should move to a status of "Success". When there are no more unprocessed events, the initial ISIM-to-IGI data load is complete. |
|
Make a few updates in the test ISIM and confirm the updates are integrated in IGI |
This step validates that after the data load, ongoing ISIM updates get processed. For example, transfer an ISIM person to a different organization. You should see the change reflected in the IGI UI. |
|
Run the ISIM-to-IGI validation tool |
As an extra means of verifying that all ISIM entities were integrated correctly into IGI, we recommend running the ISIM-to-IGI validation tool. Refer to “Appendix H: Optional ISIQ Tools” in the User’s Guide. |
|
(Optional) Run the ISIGADI-to-ISIQ migration tool |
If you’re an ISIGADI customer, you should try out the ISIGADI-to-ISIQ migration tool. It does an update-in-place of IGI entities previously sync’d by ISIGADI, and converts them to ISIQ format. The tool can save you considerable time when transitioning from ISIGADI to ISIQ. Refer to “Appendix H: Optional ISIQ Tools” in the User’s Guide. |
|
(Optional) Subscribe the test ISIM to the test IGI to create two-way integration |
For a list of IGI user events that ISIQ will forward to ISIM, refer to Appendix C of the User's Guide. Deciding whether to set up bi-directional subscriptions depends on your requirements. For some customers, a one-way flow from ISIM-to-IGI will be sufficient. |
Once you're familiar with installing/configuring ISIQ, defining products and subscriptions, and you validated ISIM-IGI data integration between your test systems, you're ready to move to Phase 2.
|
Action |
References/Comments |
|
Provision three Linux VMs for your production ISIQ to run in a three-node docker swarm cluster |
These steps assume a multi-node cluster for fault tolerance, workload balancing, rolling updates, etc. But if assigning three VMs is not practical in your environment, running a single-node configuration is also a viable alternative. |
|
Install the latest Docker CE on one of the Linux VMs; this will be the manager node in the swarm |
For enhanced fault tolerance, you might want to designate three manager nodes. For more information on this subject, see https://docs.docker.com/engine/swarm/admin_guide |
|
Download the starter kit .zip file and extract its contents on the manager node |
https://www.ibm.com/support/pages/ibm-security-information-queue-starter-kit |
|
Download ISIQ’s docker images, and create an SSL certificate for nginx |
Run <starterKitDir>/isiq setup |
|
Update <starterKitDir>/cfg/oidc/oidcSettings.json |
If you're leveraging the same OIDC provider in test and prod, you can reuse oidcSettings.json from Phase 1 with minimal changes. |
|
Apply any modifications to the default <starterKitDir>/cfg/connect/txdef.json |
Again, you can copy the txdef.json file from your Phase 1 work, assuming the data transformations are the same. |
|
Create the docker swarm |
See Step 8 in the “Summary of Installation & Configuration Steps”. |
|
Review the <starterKitDir>/yml/cluster files that deploy a three-node docker swarm |
See “Appendix A: YAML Files” in the Deployment Guide. |
|
Run `isiq start` to confirm the ISIQ docker stacks start successfully and that OIDC login works |
|
|
(Optional) Configure the logs stack |
|
|
Configure your production ISIM as a product in ISIQ |
Configuring an ISIM won’t alter its contents. To receive updates, the ISIM must be subscribed to an IGI source. |
|
Verify that all production ISIM LDAP entities are being collected and stored in ISIQ topics |
See “Verifying ISIM Data Loading” in the User’s Guide. |
|
If possible, set up a fresh IGI installation (not your production IGI) with an initialized database, and configure that IGI product in ISIQ |
You can reuse your test IGI from Phase 1. The goal is to simulate an initialized IGI in order to measure ISIQ's data load. To remove previously integrated ISIM data from IGI, see Appendix F in the User’s Guide. Once you have an empty IGI DB, it’s a good practice to take a snapshot of the IGI data tier for quick restores in case you do multiple ISIM-to-IGI test runs. |
|
Apply IGI customizations |
See Appendix A in the User’s Guide. |
|
Subscribe this IGI to your production ISIM |
This is a dry run to verify that production ISIM users, accounts, etc. are migrated across, that ISIM-to-IGI data transformations work correctly, and to note the elapsed time for the load. |
|
Log in to IGI and navigate to the “Monitor” tab |
Check “Target inbound – Account events” and the related tabs to ensure that “Unprocessed” events get processed. |
|
Determine if any production ISIM entities were not inserted into IGI |
There may be custom policies or other IGI items that need to be manually re-added. |
After any integration issues have been resolved, and you’ve completed a set of basic test cases, you're ready to move to Phase 3.
|
Action |
References/Comments |
|
Ensure you have a backup of your production IGI DB |
|
|
Apply IGI customizations to the production IGI |
See Appendix A in the User’s Guide. |
|
(Optional) Run the ISIGADI-to-ISIQ migration tool if you have ISIGADI-synchronized entries in your IGI DB |
Stop ISIGADI. Run the migration tool to convert ISIGADI entries to ISIQ format. Follow the Appendix H steps for setting initial ISIQ offsets for ISIM and IGI source topics. The purpose of setting offsets is to avoid unnecessary reprocessing of the migrated data after you define your ISIQ subscriptions. |
|
Subscribe your production IGI to your production ISIM |
|
|
Subscribe your production ISIM to your production IGI if you want two-way integration |
|
|
Use the “Monitor” tab in the IGI UI to observe the ISIM entities flowing across in the initial data load |
|
|
Re-add any missing items in the production IGI |
ISIQ offers an ISIM-to-IGI validation tool that runs in the same SDI container as the migration tool. The validation tool checks if all ISIM entities have been integrated into IGI, and reports any discrepancies. Using the report, you can determine if there are missing items in IGI that require follow-up. For details, see the validation tool write-up in Appendix H of the User’s Guide. |
|
Implement a monitoring & alerting process to verify ongoing ISIM-IGI data integration |
After the initial data load, you’ll want to establish a monitoring & alerting process that ensures ISIM updates continue to flow to IGI. ISIQ provides a sample self-monitoring framework using ELK and Grafana. It’s a starting point you can build upon, or replace with your own monitoring tools. There is also an ISIQ System Health dashboard described in the “Metrics and Alerts” section of the User’s Guide. |
* The initial data load could take many hours depending on the number of ISIM entities and the speed with which IGI processes the database inserts. As a result, the full production deployment could require more than a day. Use the elapsed time from the dry run to help with this estimate.
Document Location
Worldwide
Was this topic helpful?
Document Information
Modified date:
20 April 2023
UID
ibm10888441