Appendix B: ISIM Customizations for ISIQ

Appendix C: Mapping Entities and Events between ISIM and IGI

In ISIQ, when an IGI product subscribes to an IBM Security Identity Manager product, the various ISIM entities are loaded into corresponding entities in IGI. Conversely, when an IBM Security Identity Manager product subscribes to an IGI product, a series of IGI events are sent to initiate updates in ISIM.

This Appendix lists (1) the mapping relationships between ISIM and IGI entities, (2) the operations (add, remove, delete) that can be performed by one product on the other, and (3) the types of events that flow between the two products.

Entity-to-Entity Mapping

The following table shows how ISIM entities map to IGI entities:

For every ISIM organizational role loaded into IGI, a new application and account configuration will be created with the name, “ISIM – <isimProduct>”. For example, if you configured IBM Security Identity Manager product “myISIM6” in ISIQ, and subscribed an IGI product to “myISIM6”, the application and account configuration created in IGI will be named “ISIM – myISIM6”.

For every ISIM person loaded into IGI, if the person has an organizational role, the IGI user will be created with a corresponding external role assigned. Using the example, all organizational roles from myISIM6 will be ported to IGI as external roles in the “ISIM – myISIM6” application.

If an ISIM account has a group, an IGI account will be created under the corresponding account configuration, and a permission for that ISIM group will be assigned to the account owner in IGI.

Entity-Key Field Mapping

For every ISIM entity loaded into IGI, the DN value will be used as the key. ISIQ prefixes the keys with “<subscriberId>.<isimProduct>:”. The following table shows which ISIM key gets used for which ISIM entity, and which IGI field will contain the key:

For every service in ISIM, an application, account configuration, and events marker will be created in IGI. The same service key is stored in all three entities. However, in the IGI UI you see only the key from the events marker field (note: APPLICATION, PWDCFG, and TARGET are the names of the IGI database tables for application, account configuration, and events marker).

Events Fulfilled for IGI-to-ISIM Integration

ISIQ fulfills the following events for IGI-to-ISIM integration:

Account events:

Add account
Lock (suspend) account
Unlock (restore) account
Delete account

Entitlement change events:

Add permission to a user
Remove permission from a user
Add ISIM external role to a user
Remove ISIM external role from a user

User events:

Create a user

For the account and entitlement change events, the account owner and corresponding service must exist in ISIM. Events for users that don’t exist in ISIM will be ignored. Events for targets that don’t exist in ISIM are also ignored.

For ISIM roles, an ISIM-<productName> target is created in IGI and the external roles are created in the target. Events for the target are fulfilled by assigning the corresponding role to the user in ISIM.

The “Create User” event will be processed by ISIQ if the following conditions are met:

1. The ISIQ_IGItoISIM_FULFILL_USER_EVENTS environment variable is set to true in ISIQ’s connect-stack.yml file. By default, this environment variable is set to false. For more information about the connect-stack.yml file, see “Configuring the YAML Files” in the ISIQ Deployment Guide.
2. IGI’s EVENT_OUT_USER table exists. For more information, see “Enable the EVENT_OUT_USER table” in “Appendix A: IGI Customizations for ISIQ”.

Updating ERC Status in the IGI OUT Events Queue

After you subscribe ISIM to IGI, you will notice that the “ERC Status” column in the IGI OUT events queue shows “Unprocessed” for events sent to the ISIM that you configured, even if the events completed. To remedy this situation, check for the creation of a topic with the following name:

<subscriberID>.<isim-name>.ISIM_database.EventCompletion

Once this topic exists, go to the IGI product dashboard in the ISIQ UI and click the “Update subscription” button (for more information, see “Appendix D: Updating Subscriptions”). This action detects and reads the new EventCompletion topic. As a consequence, the “ERC Status” column in the IGI OUT events tab is updated with the result of ISIM operations. The column no longer shows “Unprocessed”.

Interpreting the Justification Field

If you configured the Justification field in the ISIM user interface, here is how to interpret the ISIQ-supplied justification data when an IGI event is sent to ISIM. For example, if you see this field:

1. Requested by ISIQ – indicates that ISIQ initiated the request.
2. isiquser.IGI – lists the product name where the request originated, so that ISIQ knows which target to update with the result from ISIM.
3. ACCOUNT – indicates whether the event came from IGI’s USER_EVENT_ERC table (labeled “ACCOUNT”) or the EVENT_OUT_USER table (labeled “USER”).
4. 64497 – specifies the request ID in the IGI OUT events table. This ID is the row number that will be updated with the result from ISIM.

Creating Custom Persons in ISIM

If you subscribe ISIM to IGI, any IGI “Create User” events will result in default Person types being created in ISIM. If you want the events to create custom Person types in ISIM, you must modify txdef.json. Specifically, find the "EVENT_OUT_USER" mapping rule under "in": "igi", "out": "isim", and add the name of your custom profile to the “txe” line as illustrated here:

  "txe": "{userid: .obj_code, operation: .operation, eventkey: .eventkey, ou: .ou, givenname: .name, dn: .dn, sn: .surname, user_erc: .user_erc, profile: \"CustomPerson\"}"

The modification consists of passing in a “profile” attribute along with the name of the profile you want the ISIM user created as. Be sure to include in the “txe” line any other required attributes for the custom Person object class that you’re using.

Ignoring IGI Account Events from a Specific User

If there are IGI account events which originate with a specific IGI user ID, and that are not needed by ISIM, you can tell ISIQ to ignore the events. To leverage the feature, perform these steps:

(1) Add the ISIQ_IGItoISIM_IGNORE_ACCOUNT_EVENT_FROM_IGIUSER environment variable to your connect-stack.yml file. The value of the environment variable has the following format:

<isimProductName>;;;<igiUserId>

The <isimProductName> is the name of the ISIM product you configured in ISIQ. The name is case-sensitive. Here’s an example of the environment variable:

ISIQ_IGItoISIM_IGNORE_ACCOUNT_EVENT_FROM_IGIUSER=isim10DC;;;iq_admin

This setting indicates that for IGI-to-ISIM integration, ISIQ will ignore account-related events submitted by IGI user “iq_admin” that are destined for the ISIM system “isim10DC”, where “isim10DC” is the ISIM product name you configured in ISIQ.

(2) Ensure the “cod_operation: .cod_operation” mapping rule is present in the "in": "igi", "out": "isim" section of txdef.json. The cod_operation attribute is essential because it contains the IGI user ID of the event requester. The default version of txdef.json already has the required mapping rule in the USER_EVENT_ERC topic transformation. You just need to verify that if you’ve modified txdef.json, your modified copy also includes this mapping rule.

Supported Operations on ISIM and IGI Entities

The following table lists ISIM entities, their corresponding IGI entities, and the operations that ISIQ supports from ISIM-to-IGI and from IGI-to-ISIM:

Notes:

1.  When an ISIM person is loaded into IGI, if that person has roles, the corresponding external roles will be assigned to the user in IGI. If a role is added to or removed from an ISIM person, a corresponding external role will be added to or removed from the IGI user.

2.  When an ISIM account is loaded into IGI, if the account has groups assigned, the corresponding permissions will be assigned to the IGI user. If a group is added to or removed from an ISIM account, a corresponding permission will be added to or removed from the IGI account.

Appendix D: Updating Subscriptions

At the time of subscription creation, ISIQ transforms data from available source topics according to the transformation rules defined in the template file, txdef.json. However, there might be cases in which new source topics get added, because of new source data, after subscription creation. These new topics are not automatically detected by the existing subscription. For example, if an ISIM user adds a new SAP adapter service profile after the IGI-to-ISIM subscription was created, the subscription won’t know about the SAP service entity topics.

To address this type of scenario, there is an “Update subscription” menu option on the ISIQ product dashboard page.

Selecting “Yes” causes the subscription to

1. Reevaluate the list of available topics and apply defined transformation rules, and
2. Subscribe to any new data that has been produced since subscription creation.

In our example, the data from the new SAP service is transformed and consumed by the subscribing product, effectively updating the subscription’s transformation rules.

There’s no harm in invoking the “Update subscription” menu option if your subscription is currently up to date. You will see this informational message box:

Here’s a related scenario to be aware of: When you subscribe an IBM Security Identity Manager product to IGI, the IGISource connector only creates topics for events it receives after the connector started, which means the IGI USER_EVENT_ERC topic won’t exist until at least one IGI account has been changed. If hours or days elapse without an IGI account change, your ISIM-to-IGI subscription won’t know that it must also monitor the USER_EVENT_ERC topic.

You can check for this scenario by running the topicList.sh script provided in the /util directory, as extracted from the ISIQ starter kit .zip file (you can also look at the “Topic List” UI described earlier in the Connectors and Topics section).

n the list of topics, search for entries belonging to your IGISource connector, such as <subscriberID>.my-igi.IGISource.EVENT_OUT_USER and <subscriberID>.my-igi.IGISource.USER_EVENT_ERC. Once you confirm those topics exist, go to the dashboard for the configured IBM Security Identity Manager product and click the “Update subscription” menu option for your IGI source. The subscription re-reads the topic list to ensure all the data is seen. Any delays in updating the subscription will not cause loss of data. It simply means that user events won’t flow from IGI to ISIM until the update action is taken.

Appendix E: Custom ISIM-to-IGI Transformations

This Appendix describes the options you have for customizing ISIM-to-IGI integration. The most common method is to update the txdef.json file. For the basic syntax and format of txdef.json, see the “Custom Transformations” section earlier in this guide. Note: Before updating txdef.json, always be sure to back up the file.

There are several use cases in which you might want to implement customized ISIM-to-IGI integration. For example, you have extended Person attributes that you want to map to IGI attributes in a particular way, or you don’t want ISIM persons to be synchronized into IGI with their default ISIM Organizational Unit (OU) names. Instead, you prefer to assign them to a specially created IGI OU. This Appendix explains how to implement those types of customizations.

In the default txdef.json, there are two sections for user transformation as shown here:

{
  "in": "Person",
  "out": "USER_ERC",
  "txe": "{pm_code: .uid[0], ou: .erparent, given_name: .givenname[0], surname: .sn[0], email: .mail[0], address: .postaladdress[0], erroles: .erroles, eralias: .eraliases, isiq_user_dn_column: \"ATTR11\", isiq_acct_owner_column: \"ATTR8\"}"
},
{
  "in": "BPPerson",
  "out": "USER_ERC",
  "txe": "{pm_code : .ercustomdisplay[0], ou: .erparent, given_name: (.cn[0]|split(\" \")|.[0]), surname: .ercustomdisplay[0], email: .mail[0], erroles: .erroles, isiq_user_dn_column: \"ATTR11\", isiq_acct_owner_column: \"ATTR8\"}"
},

As you can tell, the transformation for users must have USER_ERC as the “out” topic. Person and BPPerson are object profile names for ISIM users. After you configure an IBM Security Identity Manager product in ISIQ, you should have “Person” and “BPPerson” topics that hold person and business partner person data. For example, if you list all of your ISIQ topics, you might see:

myuserid.myISIM.directory.Person
myuserid.myISIM.directory.BPPerson

By default, the only persons that ISIQ transforms are Person and BPPerson objects. For more information about supporting custom Person and BPPerson profiles, see “How to Support Custom Persons and BPPersons” later in this Appendix.

An ISIM object profile contains information about various types of entities. The transformation section (“txe”) of txdef.json specifies the list of transformations for each attribute of an ISIM entity. The format of the transformation is:

<IGI attribute name>:<JQ filter for transforming ISIM attribute value>

The IGI attribute name cannot be changed since it’s part of a fixed set of IGI attributes. You can only customize the JQ filter for an ISIM attribute value. In other words, you can only customize which ISIM attribute to map to an IGI attribute, and how to perform the transformation between the two attributes. This same rule applies to all entity types.

In the txdef.json file, there are some IGI attribute names prefixed with er, iq, or isiq, such as erroles, iq_group_name, and isiq_user_dn_column. They are not IGI attribute names. They are used internally by ISIQ for a special purpose. In general, they must not be changed except for ones that have ATTRnn as the transformation filter. Another exception would be if you are using an adapter in which you store the unique group name in the ergroupid attribute rather than in the ergroupname attribute. In that case, change the "txe" line to set groupname to .iq_groupid instead of to .iq_groupname.

How to Customize Transformations for Accounts

Only a fixed set of IGI attributes can be specified for an account. You are limited to customizing which IBM Security Identity Manager account attribute will be transformed to each IGI attribute, and how the JQ filter performs the transformation.

Here is an excerpt from txdef.json that illustrates the default account attribute transformations:

{
  "in": "ITIMAccount",
  "out": "ACCOUNT",
  "txe": "{name: .eruid, service: .erservice, groups: .erroles, status: .eraccountstatus, owner: .owner[0], isiq_acct_owner_column: \"ATTR8\"}"
},
{
  "in": [
    "ADAccount",
    "AzureAccount",
    ...
    ...
    "ZDB2AdapterAccount"
  ],
  "out": "ACCOUNT",
  "txe": "{name: .eruid, service: .erservice, groups: .iq_groups, status: .eraccountstatus, owner: .owner[0], isiq_acct_owner_column: \"ATTR8\"}"
},

As you see, it’s a small set of IGI account attributes, and you cannot modify the IGI attribute names. There are two means of customizing this set: (1) change the ATTR8 mapping; and (2) assign values for the "NAME", "SURNAME", "DISPLAY_NAME", and "EMAIL" columns in the IGI EVENT_TARGET table.

(1) In the “Update ACCOUNT_CREATE rule flow” section of Appendix A, the EVENT_TARGET.ATTR8 column was leveraged to set the account owner DN so that the account could be matched with the account owner. If ATTR8 is already being used for another purpose in the EVENT_TARGET table, then you need to do the following:

• Decide which ATTRnn to use in the table instead of ATTR8.
• Update “Create Account - DN Matching Rule” to use ATTRnn.
• Replace all occurrences of ATTR8 with ATTRnn in txdef.json.
• Restart the ISIQ stacks.
• Update the IGI-to-ISIM subscription.

(2) The "name", "surname", "display_name", and "email" attributes are a standard part of an IGI account. However, in the default txdef.json, none of the four is present (with the exception of “name”; see the Note: below) because not all ISIM accounts contain this data, and when they do, it's often in a profile-specific source attribute that is not applicable for all customers.

To populate these attributes in your IGI accounts, you need to modify txdef.json. We recommend you separate out into its own stanza the account profile that corresponds to your ISIM account type. For example, if you are loading ISIM AD accounts into IGI, move “ADAccount” from the list of account profiles into its own stanza. The new ADAccount stanza in txdef.json might look like this:

{
"in": [
"ADAccount"
],
"out": "ACCOUNT",
"txe": "{isim_attrs: ., name: .eruid, service: .erservice, groups: .iq_groups, status: .eraccountstatus, owner: .owner[0], isiq_acct_owner_column: \"ATTR8\", display_name: .eraddisplayname, givenname: .givenname[0], surname: .sn[0], email: .eradupn}"
},

This example includes all four additional attributes, but you can specify fewer if you only require a subset. After saving the new stanza in txdef.json, restart the ISIQ stacks and update your subscription to implement the change.

Note: One possible area of confusion is the handling of the “name” account attribute. It was previously mapped to “.eruid” at the start of the “txe” list, and so ISIQ must map a different attribute to fill in the NAME column in the IGI EVENT_TARGET table. When processing accounts, the IGI sink connector uses “givenname” as a substitute for “name”. In the ADAccount example, “givenname” is mapped to “.givenname[0]” but you can choose any source attribute. The IGI sink connector sees “givenname” in the transformed record, and inserts the associated source attribute value into the NAME column.

How to Customize Transformations for Users

ISIQ leverages the USER_ERC table to create, modify or delete users in IGI. That’s why USER_ERC is defined as the “out” topic for ISIM “Person” and “BPPerson” entities in txdef.json.

ATTR1, ATTR2, and up through ATTR15 are extended attribute columns in the USER_ERC table. Some of these fifteen extended attributes are already in use, but others are available to set additional attribute values for IGI users.

Follow these steps to use ATTRnn to bring over additional ISIM person attributes to an IGI user:

1. Find out which ATTRnn extended attributes are available. The extended attributes that are not shown on IGI’s UserErc Attribute Mapping page are eligible for your use (for more information about the IGI UI, click the Help link from this page).

2. As an example, suppose you decide to use ATTR13 to set ISIM’s postaladdress attribute value. You need to add ATTR13 with your designated label on the UserErc Attribute Mapping page.

3. Update the txdef.json file. ATTR13:.postaladdress[0] must be added to the “txe” section.

{
  "in": "Person",
  "out": "USER_ERC",
  "txe": "{pm_code: .uid[0], ou: .erparent, given_name: .givenname[0], surname: .sn[0], email: .mail[0], address: .postaladdress[0], erroles: .erroles, eralias: .eraliases, isiq_user_dn_column: \"ATTR11\", isiq_acct_owner_column: \"ATTR8\", ATTR13:.postaladdress[0]}"
},

4. Restart ISIQ stacks (note: for more information about this step and the next step, see the relevant sections at the end of this Appendix).

5. Update the IGI-to-ISIM subscription.

In the Appendix A section, “Update the USER_ERC attribute mapping to set the user DN”, USER_ERC.ATTR11 is mapped to PERSON.DN. But let’s suppose that ATTR11 had been allocated for some other purpose. In that circumstance, you could switch to another available ATTRnn:

1. Update the USER_ERC Attribute Mapping page with the other ATTRnn.

2. Replace all occurrences of ATTR11 with ATTRnn in the txdef.json file.

3. Restart the ISIQ stacks.

4. Update the IGI-to-ISIM subscription.

In addition to the ATTR1-ATTR15 extended attributes in USER_ERC, any other attribute in the table can be mapped and supported by ISIQ. Consider the case in which you want to set the value of USER_TYPE in the USER_ERC table to an available attribute in the PERSON table. Since the PERSON table offers ATTR1-ATTR5 as its extended attributes, you decide to use ATTR5. To accomplish the mapping, perform these steps:

1. Verify that the attributes you want to map are available and map them to the respective fields in the Attribute Mapping UI. Note: In this screen capture, the underscore character that prefixes the _ATTR5 Label denotes a reference to the PERSON table. Without the underscore prefix, the Label specified there would simply be mapped to the IGI Name.

2. Update the txdef.json file as shown here, with “user_type” added to the “txe” section:

{
  "in": "Person",
  "out": "USER_ERC",
  "txe": "{pm_code: .uid[0], ou: .erparent, given_name: .givenname[0], surname: .sn[0], email: .mail[0], address: .postaladdress[0], erroles: .erroles, eralias: .eraliases, isiq_user_dn_column: \"ATTR11\", isiq_acct_owner_column: \"ATTR8\", user_type: \"Default Person\"}"
},

{
  "in": "BPPerson",
  "out": "USER_ERC",
  "txe": "{pm_code: .uid[0], ou: .erparent, given_name: .givenname[0], surname: .sn[0], email: .mail[0], address: .postaladdress[0], erroles: .erroles, eralias: .eraliases, isiq_user_dn_column: \"ATTR11\", isiq_acct_owner_column: \"ATTR8\", user_type: \"Business Person\"}"
},

3. Restart the ISIQ stacks.

4. Update the IGI-to-ISIM subscription.

How to Support Custom Persons and BPPersons

To allow ISIQ to transfer ISIM custom Person and BPPerson entities, follow these steps:

Step 1: Confirm your entities have been defined, and objects of that type have been created in ISIM. This is typically already the case, but if not, define the custom Person or BPPerson entity and create at least one entry of that type before moving to Step 2.

Step 2: Decide how best to update txdef.json. If you have a custom Person class, and the default Person mapping rules are applicable to it, then you simply need to add your profile name to the "in" line for the existing Person entry.

Note: If you don't know the profile name of your custom Person class, you can run topicList.sh in the /util directory to display all topics (you can also view the “Topics” link on the System Health dashboard). In the topics that ISIQ generates for ISIM, the profile name gets stored in the last level of the topic name. For example, if your custom Person class is named “IQPerson”, you would see a topic such as this:

myuserid.myISIM.directory.IQPerson

Using the IQPerson example, make these changes to txdef.json:

    "in": "Person",
    "out": "USER_ERC",

to:

        "in": [ "Person", "IQPerson" ],
    "out": "USER_ERC",

Notice how the "in" line was converted from a JSON string to a JSON array because there is now more than one type of Person. If you have other custom Person classes that use the same transformation rule, add them as other comma-separated entries in this array.

However, if your custom Person class requires a different mapping relationship between ISIM and IGI attributes, you need to create a new entry in the txdef.json file. Copy the lines for Person, paste them after the end of the Person entry, then customize as needed. The full entry might look like this:

      }, // End of the Person entry
      {
        "in": "IQPerson",
        "out": "USER_ERC",
        "txe": "{pm_code: .employeeNumber, ou: .erparent, given_name: .givenname[0], surname: .sn[0], email: .companymail[0], address: .postaladdress[0], erroles: .erroles, eralias: .eraliases, isiq_user_dn_column: \"ATTR11\", isiq_acct_owner_column: \"ATTR8\"}"
      },

The full entry includes the open brace “{" through the closing brace and comma "},". In this example, the line “}, // End of the Person entry” was included to illustrate how to append your custom Person entry after the default Person entry.

The process operates the same when working with a custom BPPerson. Just use the default BPPerson entry in txdef.json as the starting point for your modifications.

Step 3: Once the custom person data exists in ISIM and the txdef.json file has been updated with the appropriate mapping, ISIQ must discover this new custom person data, which requires

How to Support a Custom Service

In ISIM, if you have a custom service type, you need to update the txdef.json file so that ISIQ can load the services, groups, and accounts for that custom service type into IGI. Here are the ISIQ steps for supporting a custom service:

1. Find the object profile name for the service profile, group profile, and account profile.
2. Update txdef.json with the new profile names.
3. Restart ISIQ stacks.
4. Update the IGI-to-ISIM subscription.

The following example uses the sample command-line service type to demonstrate the customization process. The example was developed by using the Command Line (CLIx) Adapter (refer to https://www.ibm.com/support/pages/verify-adapters-product-documentation-pdfs).

Find Object Profile Name for Service, Group, and Account

In ISIM, an object profile represents different types of ISIM entities. There can be different profiles in the same entity type category. For example, the POSIX AIX service profile and POSIX Linux service profile are two different service profiles in the Service category. The POSIX AIX account profile and POSIX Linux account profile are two different profiles in the Account category.

After an adapter profile JAR file is imported into ISIM, the adapter’s service profile, group profile, and account profile are created in the LDAP server. A profile has information about its respective service, group, or account. It also has information about the relationship with the other two types.

Every adapter has a service profile and account profile. Some adapters do not have a group profile if groups are not managed by the adapter. Some adapters (such as the POSIX AIX adapter) have multiple group profiles if multiple group types are managed by the adapter.

If you are familiar with service profiles, you can find the service, group, and account profile names from the LDAP directory server as shown here:

For this sample CLIx service type, SampleCmdProfile is the service profile name, SampleCmdAccount is the account profile name, and SampleCmdGroupProfile is the group profile name.

If you are not familiar with ISIM service profiles, follow these steps to find the service, group, and account profile names for your custom service type:

3. From the service type form, select the *Service tab, and write down the value for LDAP class, erSampleCmdService.

4. From the service type form, select the *Account tab, and write down the value for LDAP Class, erSampleCmdAccount.

5. From the service type form, select the Group tab, select the group name, write down the LDAP class value, ersamplecmdgroups.

If your custom service has multiple group types, then you need to find out the LDAP class value for each group type. As an example, if you look at the POSIX AIX profile (service type), it has two group types: one for AIX group and another one for AIX role.

6. From the LDAP directory server, search the service profile which has erSampleCmdService for its ercustomclass value, as shown in the next screen capture. The Search DN is set to the base DN where all service profiles can be located. The “ou=org,DC=COM” substring represents the tenant DN. Your tenant DN might be different than what is shown in this example:

The object profile name (erobjectprofilename) value for this service type is SampleCmdProfile. The screen capture shows erobjectprofilename as being part of the DN.

7. Find the account profile name using the account LDAP class, erSampleCmdAccount. The account profile name is SampleCmdAccount.

8. Find the group profile name using the group LDAP class, ersamplecmdgroups. The group profile name is SampleCmdGroupProfile.

You have now found service, group, and account profile names as shown here:

• Service profile name: SampleCmdProfile
• Group profile name:  SampleCmdGroupProfile

• Account profile name: SampleCmdAccount

Update txdef.json with the new profile names

Save a copy of <isiqStarterKit>/cfg/connect/txdef.json before you modify the file. Update txdef.json with the service, group, and account profile names:

1. In the ISIM-to-IGI section, find the section for service entity mapping and add the service profile name to the input (“in”) topic list. The ISIM-to-IGI section is the section where it has “in”: “isim” and “out”: “igi”. The service entity-mapping section is where it has “out”: “SERVICE”. You may see multiple sections where it has SERVICE as output. The ones with multiple service profiles are the ones using the default transformation. If the default transformation for the service is not what you want for your custom service, you might need to have a separate section for your service profile. For this sample CLI service type, add the service profile name SampleCmdProfile at the end of the service profile list for the default transformation.

2. In the ISIM-to-IGI section, find the section for the group entity mapping and add the group profile name. If the default transformation for the group is not what you want for your custom group, you might need to have a separate section for your group profile. For this sample CLI service type, add the group profile name SampleCmdGroupProfile at the end of the list for the default transformation. If you have multiple group types in your custom service type, you need to add each one of them to the list.

3. In the ISIM-to-IGI section, find the section for account entity mapping and add the account profile name. If the default transformation for the account is not what you want for your custom account, you might need to have a separate section for your account profile. For this sample CLI service type, add the account profile name SampleCmdAccount at the end of the list for the default transformation.

For the last two steps in the customization process, see “Restart the ISIQ stacks” and “Update the IGI-to-ISIM Subscription” at the end of this Appendix.

How to Skip OU Synchronization

By default, if ISIM persons belong to an Organizational Unit called “ISIM_OU1”, ISIQ will synchronize those persons into IGI as users in an OU called “ISIM_OU1”. However, you may prefer to synchronize them into, say, “My_IGI_OU”. The following instructions explain how to skip the default OU sync mechanism and instead implement a different procedure.

1. Choose an ISIM person attribute that will contain the preferred IGI OU code. In this example, we will add an "ou" attribute in ISIM by using the “Design Forms” feature:

As an attribute of organizationalPerson, “ou” will be available to Person, BPPerson, custom Person, and custom BPPerson objects.

2. Update ISIQ's connect-stack.yml file by inserting the following environment variables under
“services:”
  ”connect:”
    ”environment:”:

ISIQ_SKIP_OU_SYNC_ISIM_PERSON_OU_ATTR=jsong0104.btisim6:ou

Specifies which person attribute will contain the OU code in IGI. In our example, the attribute was called “ou”. The value of this environment variable must start with an ISIM key prefix in the format,“<subscriberID>.<isimProductName>:“.

ISIQ_SKIP_OU_SYNC_ISIMtoIGI_DEFAULT_OU_CODE=jsong0104.btisim6:igiDefaultOu

Specifies a default OU in case the ISIM person has an invalid OU.

ISIQ_SKIP_OU_SYNC_IGItoISIM_DEFAULT_OU_DN=jsong0104.btisim6:erglobalid=8354189953611274681,ou=orgChart,erglobalid=00000000000000000000,ou=org,dc=com

Specifies the OU DN in ISIM that will be assigned after a new user is created in IGI.

For these environment variables, you can provide multiple values by using three semicolon characters (;;;) as a delimiter. For example, if you have two IBM Security Identity Manager products, and you want to implement the same “Skip OU Synchronization” behavior in both of them, here's what you specify:

ISIQ_SKIP_OU_SYNC_ISIM_PERSON_OU_ATTR=mySubscriberID.isim1:ou;;;mySubscriberID.isim2:ou

If you have a two-way subscription between ISIM and IGI, any new IGI users get synced to ISIM into the OU that you specified in the ISIQ_SKIP_OU_SYNC_IGItoISIM_DEFAULT_OU_DN environment variable. When a new person is created in ISIM, its “ou” attribute is set according to that environment variable.

3. Update txdef.json file so that IGI's OU column in the USER_ERC table can be mapped to the ISIM person's “ou” attribute. By default, your txdef.json will contain "ou: .erparent". The following excerpt shows the change that you make (highlighted in black) for “Person”. Because you must have a common attribute for all Person types, you need to repeat this change for all Person entities in txdef.json. Since "ou" is a multi-valued attribute, notice that it uses the array format, .ou[0]:

{
        "in": ["Person"],
        "out": "USER_ERC",
        "txe": "{pm_code: .uid[0], ou: .ou[0], given_name: .givenname[0], surname: .sn[0], email: .mail[0], address: .postaladdress[0], erroles: .erroles, eralias: .eraliases, isiq_user_dn_column: \"ATTR11\", isiq_acct_owner_column: \"ATTR8\", ATTR12: .employeenumber}"

},

To ensure that no IBM Security Identity Manager OUs get loaded into IGI, remove all txdef.json sections for the different types of OUs, specifically any section whose “out” value is OU_ERC, as shown here:

      {
        "in": "Organization",
        "out": "OU_ERC",
        "txe": "{name: .o[0], description: .description[0]}"
      },
      {
        "in": [
          "AdminDomain",
          "BPOrganization",
          "Location",
          "OrganizationalUnit"
        ],
        "out": "OU_ERC",
        "txe": "{name: (.ou[0]//.l[0]), description: .description[0], erparent: .erparent, ersupervisor: (.eradministrator[0]//.ersponsor[0]//.ersupervisor)}"
      },

Refer to the “Custom Transformations” section in this User’s Guide for txdef.json syntax.

4. On IGI’s Access Governance Core -> Manage -> Groups page, make sure that the OU’s “ID Code” matches the <subscriberID>.<isimProductName>:<igioucode> format.

The ID Code must be lowercase. If you specify mixed case such as “Jsong0104.Btisim6:igiOu01”, ISIQ cannot match it to the lowercase value set from ISIM's “ou” attribute.

5. Once you’ve defined a person attribute to hold the IGI OU code, you need to make sure each ISIM person has the OU code, which is “igiou01” in this screen capture:

6. Implement your “Skip OU Synchronization” changes by performing these actions:

a) Remove the ISIQ app stack first and then remove the connect stack.

b) Deploy the connect stack and then deploy the app stack.

c) Use the “Reprocess” button on the ISIQ product's Subscription page to select the “Person” topic and reload ISIM persons. With the “ou” attribute created, the environment variables specified in connect-stack.yml, and the txdef.json updates, all ISIM persons will be placed in the preferred IGI OU if the persons have an “ou” attribute value that matches IGI’s ID code.

Additional ISIM Customization Options

The <starterKitDir>/cfg/connect/isimConfiguration.json file offers additional options for customizing how ISIM data is read or skipped. Because the file is a JSON array, you can use it to customize multiple ISIM servers. In most cases, only one ISIM server is configured in ISIQ, and thus the default “global” tenant in isimConfiguration.json is sufficient. But if you configured more than one ISIM in ISIQ, you can define each one in isimConfiguration.json with a corresponding “tenant” DN value. When adding a non-global “tenant” definition, be aware of these factors:

(1) It should be the same DN you specified as “Tenant DN” when you configured the IBM Security Identity Manager product in ISIQ. You must also prepend the DN with the LDAP host, port number, and a ‘/’ delimiter, for example:
      “tenant”: “ldapHostA:389/ou=org,dc=com”,

(2) There’s no inheritance from the global tenant, in other words, the customizations don’t begin with the global settings and then override them with any that you specify for a non-global tenant. ISIQ uses the settings for whichever entry matches. You can either default to the global tenant, or you can specify your own tenant field with a DN corresponding to your IBM Security Identity Manager product and use its settings in place of the global tenant.

Here is a list of the customizations you can implement with isimConfiguration.json:

Support Auxiliary Object Classes

When you create custom entities in ISIM, you define them using a combination of structural and auxiliary object classes. ISIQ only knows about attributes in the default ISIM object classes and in the “ercustomclass” object class. If extra ISIM attributes are defined using additional auxiliary object classes, the extra attributes will be rejected by ISIQ as invalid since they belong to unknown classes.

To get around this restriction, the isimConfiguration.json file allows you to specify one or more object classes that should be used for a given ercustomclass. As an example, note the "auxClasses" field listed at the end of the default isimConfiguration.json:

              "auxClasses": {
              "OrganizationalUnit": [ "iqExtraOrg" ]
      },

These statements define an "iqExtraOrg" auxiliary object class so that ISIQ can successfully load its attributes into Kafka topics.

If you see Data Exception errors in your ISIQ connect service log, such as

companyname is not a valid field name

And you know you have an attribute called “companyname” defined in an ISIM custom OU, then you need to add the name of that custom OU as an auxiliary object class to the “OrganizationalUnit” section of isimConfiguration.json.

Configure LDAP Connection Pool

When you configure an IBM Security Identity Manager product in ISIQ, the most frequent reads are performed against the LDAP directory. To speed data integration, the ISIQ directory source connector starts ten worker threads to run multiple LDAP reads in parallel, and so pooling is helpful to manage the multiple connections.

ISIQ uses the Apache LDAP implementation, which supplies a set of configuration parameters for its connection pool. To learn more about the parameters, select the “GenericObjectPool” class on the Javadoc page for the Apache Commons API:

https://commons.apache.org/proper/commons-pool/apidocs/index.html

In isimConfiguration.json, the “ldapPool” field under “tenant”: “global” lists default values for the connection pool parameters. The defaults are adequate for most purposes, but you are free to override them in “global”, or in your own “tenant” definitions. Most of the connection pool parameters can be understood from the Javadoc page. However, there are two that need more explanation:

(1) “whenExhaustedAction”: This setting determines what action to take when there are no available connections. The setting can have one of three possible numeric codes:
0 = Immediately fail with an exception.
1 = Wait for maxWaitForConnectionMillis to see if a connection becomes available before failing with an exception.
2 = Create a new connection when no more are available.

(2) “searchTimeoutMillis”: This is the time in milliseconds that the directory source connector will wait for an LDAP query to complete. The “searchTimeoutMillis” field is not an Apache LDAP connection pool setting. It was previously a global environment variable in ISIQ’s connect stack, but since it belongs with other LDAP configuration information, it was moved under “ldapPool” where it can be customized at the tenant-level if you choose.

Configure LDAP Search Limit and Lookahead Values

By default, for each query to the LDAP directory, the ISIQ directory source connector attempts to retrieve 1000 objects, known as the search limit. To avoid missing any objects that have the same modify timestamp and that cross a search-limit boundary, the source connector buffers the next 100 objects. This is known as the lookahead value. Setting it to 10-20% of the ldapSearchLimit is usually appropriate.

In most circumstances, these defaults are adequate. However, you might have, for example, an LDAP directory that’s not responding quickly. As a result, the recurring queries for 1000 objects take too long to complete, causing Kafka timeouts and other unwanted side effects.

To address these types of situations, you can customize the search limit and lookahead values used by the directory source connector, as shown here:

              "ldapSearchLimit": 500,
              "ldapLookAhead": 50,

In this example, the defaults are reduced to accommodate a slower-performing directory.

Configure LDAP Poll Delay Interval

By default, the ISIQ directory source connector waits 5 seconds before polling for the next batch of directory modifications. If you are concerned about LDAP system overhead and want to slow the process down, you can increase “pollDelaySeconds” above the default 5 seconds. The maximum allowed value is 43200 or 12 hours. The disadvantage of setting a large pollDelaySeconds value is that ISIQ can no longer obtain timely access to LDAP directory modifications.

As an illustration, let’s suppose you’re diagnosing a high-CPU problem and wish to temporarily reduce the activity of the ISIQ directory source connector. You decide to set the connector to wait 15 minutes between each polling interval. In that case, add the following parameter to isimConfiguration.json:

              "pollDelaySeconds": 900,

Once the high-CPU problem has been diagnosed, delete the “pollDelaySeconds” parameter and allow the default poll delay value to go back into effect.

Map Embedded ISIM Attribute to IGI Attribute

ISIQ attribute mapping generally operates in a direct manner where ISIM_AttributeA gets assigned to IGI_AttributeA. But occasionally there are scenarios where the mapping relationship is not quite as straightforward, where you want to use ISIM_AttributeA—which is embedded within an ISIM object—as a means of looking up ISIM_AttributeB. You then want to assign ISIM_AttributeB to IGI_AttributeA. To handle situations of this type, you can add one or more lookup definitions in the "embeddedAttributes" section of isimConfiguration.json, for example:

              "embeddedAttributes": {
                             "Person": [ "manager" ]
              },

The format is <profilename>:<attribute array to look up>. The first parameter is an ISIM object. The second parameter is treated as an array because you might want to look up multiple attributes. Both the profilename and the attribute array are converted to lowercase, and so it doesn't matter how you specify their case in the definition. The one requirement when using this feature is that the attribute array MUST contain a DN. If not, you will encounter an error indicating there was an invalid DN and the data for the attribute lookup won’t be filled in.

When would you want to use "embeddedAttributes"? A typical scenario involves the ISIM Manager attribute. It's often assigned as an attribute of a Person object. By using “embeddedAttributes”, you could implement the following rule when integrating ISIM data to IGI:

"For every Person object, if it has a Manager attribute value, assign that value to IGI ATTR1."

Besides adding the definition in isimConfiguration.json, you must also code a mapping rule in txdef.json. If we continue the IGI ATTR1 example, the rule might look like this:

ATTR1: ._embedded.manager[0].cn[0]

The format is <IGI ATTR>: ._embedded.<attributeName>[x].<referencedObjectAttribute>

For <attributeName>[x] you can specify something other than array element 0 if it's a multi-value attribute. If it's a single-value attribute, just specify [0]. But you must use the [ ] syntax because the embedded attribute is treated as an array.

Block Modify Account Events to IGI

After the initial load of ISIM data into IGI, you will notice a repeated set of Modify Account events in IGI’s “TARGET inbound - Account events” queue. These events originate with an automated service account. When you configure an IBM Security Identity Manager product in ISIQ, it is common to specify “itim manager” as the “WebServices” user. After you complete product configuration, ISIQ attempts to log in to ISIM every few minutes, using the “itim manager” account, to obtain the status of the web services interface. Each login causes the lastLoginTime attribute for “itim manager” to be updated, which in turn generates a Modify Account event that gets sent to subscribers such as IGI.

If you want ISIQ to block some or all of these Modify Account events from going to IGI, set the "blockMode" and “blockAccounts” fields in isimConfiguration.json. At the end of the default isimConfiguration.json that’s shipped with ISIQ, you will see the following name-value pairs:

        "blockMode": "1",
        "blockAccounts": [
                "itim manager"
        ],

In “blockAccounts”, you list one or more accounts whose updates will not always trigger a Modify Account event. There are three options for “blockMode”, signified by a different numeric code:

0 = All events for this account will be blocked.

1 = The first event after an ISIQ restart will be sent, the rest will be blocked.

2,X = Only process an event for this account if X minutes have elapsed since the last update.

Be sure to place the "blockMode" and “blockAccounts” fields under the tenant—whether it’s “global” or a specific tenant—whose Modify Account events you want to suppress.

Reduce ISIM Database Queries

By default, the ISIM database source connector queries for more than a dozen categories of audit events from the AUDIT_EVENT table in the ISIM database. Currently, audit event records are not used when IGI subscribes to ISIM. If you don't have another ISIM subscriber, such as an external application, that consumes audit event records, you can reduce the overhead of the ISIM DB source connector by shrinking the list of events it queries for.

The "dbTables" field at the end of the default isimConfiguration.json demonstrates how you would specify that only Reconciliation events should be read by the ISIM DB source connector, and all other event categories skipped. If at a later time you need to collect a wider variety of audit events, you can add them back into the "dbTables" list.

If you are in fact consuming many categories of audit events, we recommend you create the ISIM DB indexes listed in the “Tuning Requirements” section of the ISIQ Deployment Guide.

Exclude or Include Services

When a product like IGI subscribes to an ISIM data source, all services are sent. If you want your subscriber to receive only a subset of ISIM services, you must fill in either the "excludeServices" or “includeServices” field in isimConfiguration.json, as illustrated here:

              "excludeServices": [

                             "4164614613545430761",
                             "2874278754862299112"
              ],
              "includeServices": [
              ],

Enter a comma-separated list of service erglobalids in the desired field. You should provide values for either “excludeServices” or “includeServices”, but not both. If you provide both, “includeServices” will get used. The rule of thumb is to specify whichever field allows for a shorter list. If there are a small number of services to exclude, create an “excludeServices” list. On the other hand, if there are only a few services that you want ISIM to send to IGI, create an “includeServices” list. You should also consider what will happen if new services are added in the future. Generally, an “excludeServices” list would be less likely to require changing, assuming that you want to integrate the new services to IGI.

Note: Because “ITIM Service” is a reserved ISIM service, you cannot specify it in an “excludeServices” list. Conversely, if you create an “includeServices” list, it must contain “ITIM Service” in its erglobalid value as "00000000000000000002".

Implement a Naming Attribute Map

The “uid” attribute is important in ISIQ because it serves as a key to achieve parallelism across multiple Kafka partitions. By default, ISIQ uses “uid” for Person and Business Person profiles. If you don’t use “uid” for person objects, ISIQ will store all the objects in the same partition, which slows throughput.

To avoid this situation, implement a naming attribute map. In the map, you specify the ercustomclass for your person objects and the naming attribute that ISIQ should look at when processing those objects, as shown in this example:

              “namingAttrMap": [

                             "iqperson”: “givenname”
              ],

Because you’re using “givenname” instead of “uid”, you map “givenname” to the ercustomclass attribute “iqperson”. As a result, ISIQ will leverage “givenname” in the same manner as “uid” and thereby gain the performance benefits of Kafka partitions.

Implement a Profile Topic Map

The ISIM source connector creates a Kafka topic for each non-empty object profile it finds in the LDAP directory. By default, the topic name is assigned based on the object profile name. Due to the limited Latin character set that Kafka supports, it’s possible to have ISIM object profiles whose names contain characters disallowed by Kafka.

To circumvent this restriction, you can implement a profile topic map. First, identify any ISIM object profile whose name includes characters other than “a-zA-Z0-9” and “-“. Next, find the ercustomclass attribute for that object profile. Map the ercustomclass attribute to the name of the topic you want the objects to be placed in. Here’s an example:

              “profileTopicMap": [

                             "customProfileClass”: “MyCustomProfile”
              ],

The ercustomclass attribute “customProfileClass” is associated with an object profile whose name can’t be used as is when constructing a topic. And so you map “customProfileClass” to “MyCustomProfile” (the topic name you specify is case-sensitive). This means that any objects which ISIQ finds for “customProfileClass” will be placed in the “MyCustomProfile” topic for subscribers to consume. Also, because you’ve defined a mapping relationship between “customProfileClass” and “MyCustomProfile”, you must add this mapping to the txdef.json file, as described earlier in this Appendix.

Restart the ISIQ Stacks

After you make changes in isimConfiguration.json or txdef.json, you must restart the ISIQ stacks to implement the changes. A restart entails removing all stacks and then deploying them again. The commands for restarting can be found in the ISIQ Deployment Guide sections, “Removing the ISIQ stacks” and “Deploying the ISIQ stacks”.

Update the IGI-to-ISIM Subscription

After you make changes in txdef.json and restart the ISIQ stacks, you also need to update the IGI-to-ISIM subscription so that custom service-related entities are loaded into IGI. Follow these steps:

1. From the ISIQ UI, select your IGI product from the Control Center menu. You must navigate to your IGI product dashboard since you want to make the custom service-related topics from ISIM available to your IGI subscriber.

2. Go to the “Subscriptions” half of the product dashboard, find the IBM Security Identity Manager product that IGI is subscribed to, and then perform one or both of these update actions:

   (a) If there’s a new ISIM profile with associated topics that you want to load into IGI, click the “Update subscription” button (  ). That forces a re-read of all ISIM topics.

   (b) If you’ve changed the transformation rules for Person or BPPerson entities, click the “Reprocess” button  and then select the “*.Person” or “*.BPPerson” topics for reprocessing. That causes ISIM Person-related topic data to get replayed using your changed transformation rules when inserting the entities into IGI.

Appendix F: Removing ISIM Data in IGI

If you have an ISIQ environment that was used for ISIM-to-IGI data integration, and the environment gets reset (refer to the “Docker Cleanup” section of the ISIQ Deployment Guide), you might want to first remove previously integrated ISIM data from your IGI database in order to start over. A similar scenario occurs if you’re testing a custom transformation change in txdef.json and you want to clear out the IGI DB before performing a full ISIM-to-IGI data load to validate the txdef.json change. Note: These scenarios typically only arise in test and QA environments, and not in production.

Recommendations/Caveats:

(1) The instructions in this Appendix assume you have a small-to-moderate number of entities to delete from IGI. If instead you have tens of thousands of users associated with an application and you attempt to delete the application in the IGI UI, the UI can hang while the background processing occurs. When you perform high-volume deletes, it is recommended that you not attempt to do so in the UI. One alternative is to use IGI’s Bulk Data Load tool to remove large numbers of users, applications, etc. As an example, see the IBM article, Remove Users Record Track. The Bulk Load tool can be used in conjunction with the IGI Export Report feature to run queries that produce files, which become input to the Bulk Load tool. For more information, see “Export reports” in IGI’s Available reports.

(2) If you’re migrating from ISIGADI to ISIQ, you should not use these instructions to remove ISIGADI-synced entities from IGI before switching to ISIQ, and especially not if it’s a production IGI. There’s a risk that some of your ISIGADI-based customizations or other manual changes in IGI will be lost in the process. When migrating, it’s better to use the ISIGADI-to-ISIQ migration tool described in Appendix H.

(3) If you don’t want the deletes in IGI to generate OUT events that flow back to ISIM, refer to the instructions in the IGI troubleshooting technote, How to execute operations on Identity Governance & Intelligence (IGI), without triggering new events.

If you can revert to a snapshot of an empty, initialized IGI data tier, then that’s the quickest way to remove previously integrated ISIM data. But if you don’t have a usable snapshot, here is the sequence to follow when manually deleting IGI data that was ported from ISIM:

1. Applications
2. Account Configurations
3. Rights Lookups
4. Users
5. Organizational Units
6. Events for Account, Access, User, and Organizational Unit

The detailed steps are shown in the following sections.

Log in as Admin User

Log in to the IGI Administration Console as an administrator user. Note: By default, when a user is created in IGI, it’s a Service Center user. The ISIM data removal steps require an administrator user. For help with assigning an administrator role to an IGI user, refer to the appropriate IBM Documentation links, for example: User administration.

Remove Applications

• Go to Access Governance Core.
• Go to Manage > Applications.
• Select all the applications that were ported from ISIM.

• Select Remove from the Actions menu.

Remove Account Configurations

When a service is ported from ISIM, an application and an account configuration are created, one per service. So, whenever you remove the application from IGI, you must also remove its account configuration.

• Go to Access Governance Core.
• Go to Manage > Account Configurations.
• Select all account configurations that were created for ISIM services.

• Select Remove from the Actions drop-down menu.

Remove Rights Lookups

When a permission with rights is ported from ISIM, a canonical list of lookup values is created. Thus, whenever you remove an application with this type of permission from IGI, you must also remove its Rights Lookup configuration:

• Go to Access Governance Core.
• Go to Configure > Lookup.
• One by one, select each Right and then select Remove from the Actions drop-down menu.

NOTE: There are three reserved entries that must not be removed: 1_VPN, 2_EMAIL, 3_PRODUCTLINE.

Remove Users

• Go to Access Governance Core.
• Go to Manage > Users.
• Select all users that were ported from ISIM.
• Select Remove from the Actions drop-down menu.

NOTE: The sequence of operations matters here. If Organizational Units are removed before Users, then all users will disappear from the IGI UI, but they will remain in the IGI database.

Remove Organizational Units

• Go to Access Governance Core.
• Go to Manage > Groups.
• Select the parent OU that was ported from ISIM.
• Select Remove from the Actions drop-down menu.
• Select Remove Node and Children from the confirmation window.
• Select OK.

NOTE: After IGI organizational units are deleted from the IGI administration console, you need to also delete the OUs from the ORGANIZATIONAL_UNIT_ERC table. Delete only the ones ported over from ISIM. You can identify them by the <subscriberID>.<configurationName>: prefix in the OU column. For example, if the ISIQ subscriber ID was “scharle” and the configured IBM Security Identity Manager product name was “Alpha”, then an SQL select command such as

select id, name, ou from igacore.organizational_unit_erc where ou like 'scharle.Alpha%';

shows the following:

Then, run a Delete SQL command to remove just those rows.

Remove Events

• Go to Access Governance Core.
• Go to Monitor -> Target inbound – Account events.
• Select all account events for the application ported from ISIM and remove them.
  • Go to Actions -> Remove.
  • If you want to remove all the ones shown on this page, go to Actions -> Select Delete all filtered.
• Also remove events for that ISIM in the following tabs:
  • Target inbound – Access events
  • IN – User events
  • IN – Org. Unit events

Once you’ve completed the deletes and have an empty IGI database, it’s a good practice to take a snapshot or backup of the IGI data tier. This will allow you to quickly restore the initialized environment in case you need to rerun the ISIM-to-IGI data load.

Reloading ISIM Data

After you restore an initialized IGI database and you’re ready to reload ISIM data, you have two choices:

1. Run `docker volume prune --all`. This is a powerful command that should be used with caution. It erases all persistent ISIQ data, including product configurations, subscriptions, and Kafka topic information. You then have to reconfigure your ISIM and IGI products, and re-subscribe IGI to ISIM. Doing so causes another data load to occur. For a fuller discussion of `docker volume prune -all`, see the “Docker Cleanup” section in the ISIQ Deployment Guide.
2. A more granular approach is to run the <starterKitDir>/util/topicList.sh script to list all topics. Next, run <starterKitDir>/util/topicDelete.sh <topic_name> to remove each topic belonging to the IGI product you want to reload. These topics have “IGI_database” in their name (note: you might need to stop ISIQ’s app and connect stacks before the topicDelete.sh command takes effect, if the connect stack still has a hold on the IGI topics). Then, go to the IGI product dashboard page and click the “Reprocess” button for your IGI-to-ISIM subscription. Select all topics for reprocessing. By performing these steps, you replay directory source topics from the beginning, which in effect reloads your ISIM data into IGI.

Appendix G: ISIM Integration with Cloud Identity

ISIQ supports an experimental ISIM-to-IBM Cloud Identity (CI) integration in which ISIM Persons are imported into CI as Users. There are default transformation rules defined in txdef.json that map ISIM Person and BPPerson entities to CI Users.

Note: Since this ISIQ feature was introduced, IBM Cloud Identity has been rebranded as IBM Security Verify, and so to learn more about IBM’s Identity-as-a-Service (IDaaS) offering, search the IBM Docs for “Security Verify”.

Here are the steps to follow to try out this integration:

1) Navigate to the “Configuration” page in your CI instance.

2) Click the “Add API Client” button and register ISIQ as an API Client. Grant ISIQ access to All APIs by toggling “Select All”. Select “Save” when finished.

3) Edit the created API Client instance (using the Pencil icon) and take note of the “Client ID” and “Client Secret” fields.

4) Configure your application as an ISIQ product by choosing “IBM Cloud Identity (experimental)” from the “Select a Product” page:

5) Complete the required fields, with values noted from Step 3)

After you add your Client Secret and click “Configure”, your CI product is registered with ISIQ.

6) Navigate to the ISIQ dashboard of your CI product (called “my-ci-instance” in this example) and create a subscription to your configured IBM Security Identity Manager product.

7) Navigate to “Users & groups” in your CI instance to verify that ISIM Persons are being inserted.

Note: ISIM-to-CI is currently a one-way integration that imports ISIM Persons as CI Users. Other entity integrations and operations between CI and ISIM are not yet supported.

Appendix H: Optional ISIQ Tools

This Appendix describes optional tools that ISIQ offers to let you perform assorted utility functions. The tools are available in the isiq-sdi-tool zip file that you can download from the ISIQ Starter Kit Page.

ISIM-to-IGI Validation Tool

After you subscribe IGI to ISIM, you will likely want to validate whether all ISIM entities (services, groups, users, accounts and roles) were loaded successfully into IGI. To satisfy this requirement, ISIQ provides a containerized IBM Security Directory Integrator (SDI) server that runs assembly lines to check if each ISIM entity exists in IGI with the correct key. The results are reported in docker log messages so that you can validate if data integration between the products is complete.

The steps for configuring and running the validation tool are explained in

<starterKitDir>/isiq-sdi-tool/README.txt

If the tool finds that any ISIM entities were not integrated into IGI, refer to the ISIQ diagnostic procedures described in the ISIQ Troubleshooting Guide.

ISIGADI-to-ISIQ Migration Tool

To help ISIGADI users migrate to ISIQ, there is a migration tool that runs, like the validation tool, in the containerized SDI. The procedure for configuring and running the migration tool is described in

<starterKitDir>/isiq-sdi-tool/README.migrate.txt

One of the last steps in the procedure is to “Set ISIM LDAP source offset and IGI DB source offset.” The reason for performing this step is the following: Suppose you have 200K ISIM accounts that ISIGADI loaded into IGI. You stop ISIGADI, run the migration tool to convert the 200K accounts to ISIQ format, and then you subscribe IGI to ISIM from the ISIQ UI. By default, ISIQ will process all of those accounts again because from the standpoint of the Kafka-based Account source topic, ISIQ is starting at offset 0, which means it reads LDAP from the beginning. Consequently, a lot of unnecessary overhead occurs as ISIQ reprocesses account data already stored in IGI.

To prevent this scenario from happening, you should determine the timestamp of the last ISIGADI entry which was sync’d into IGI, and then specify that timestamp as an offset in ISIQ. Configuring a timestamp offset causes ISIQ to begin processing ISIM add/change/delete events after the offset you specify.

The same mechanism works for integrating IGI events to ISIM after running the migration tool. In that circumstance, you don’t want the same IGI events to be integrated to ISIM again, and so you set offsets equal to the last Event ID numbers that were processed.

Here are the instructions for setting offsets:

Setting Initial Offsets for ISIM and IGI Source Topics

As of ISIQ 1.0.7, you can set initial offsets for ISIM and IGI source topics. The offsets are used in the ISIQ connector logic to indicate which entities have previously been processed for a topic. The format of the offset will vary depending on the entity type.

For ISIM-to-IGI integration, the LDAP timestamp of ISIM entities is used as the offset. The LDAP timestamp must be in the format of yyyymmddhhmmssZ, where “yyyymmdd” is the date using a 4-digit year, 2-digit month, and 2-digit day; and “hhmmss” is the time using 2-digit hour, 2-digit minute, and 2-digit seconds. The timestamp must be in GMT or UTC time, also known as Zero Meridian or Coordinated Universal Time. For example, if you are located in the Pacific time zone, which is UTC-8, and you want to set the offset to the current time, add 8 hours to the current time. You also need to set the date accordingly. When this offset is specified, only ISIM entities created or updated AFTER that timestamp will be populated in LDAP source topics to be consumed by IGI.

For IGI-to-ISIM integration, the IGI event’s unique ID is the offset. The unique ID is the integer that is specified as the primary key in IGI’s event tables. When integrating IGI events to ISIM, ISIQ reads account events and user events. Account events come from the USER_EVENT_ERC table, and user events come from the EVENT_OUT_USER table. When you specify the offset in ISIQ, only IGI events that have an ID GREATER than the offset ID will be populated in the IGI source topics. For example, if you set the offset ID as 2999, then all the events with ID 3000 or greater will be populated in IGI source topics to be consumed by ISIM.

To set the initial offsets for ISIM and IGI source topics, you must set the following environment variables in your connect-stack.yml file:

• IQ_ISIM_LDAP_OFFSETS_AFTER_ISIGADI_MIGRATION
• IQ_IGISRC_USER_EVENT_ERC_OFFSET
• IQ_IGISRC_EVENT_OUT_USER_OFFSET

If you have a cluster environment, set these environment variables, with the same values, in all of your connect-stack.yml files.

These environment variable offsets are read if and only if non-zero offsets do not exist in the Kafka offset topics. In other words, environment variable offsets are used only once after ISIQ’s connect stack is started. After new entities begin to be processed from a source topic, the offset of the last entity processed becomes the new offset in the Kafka offset topic. When that occurs, the offset environment variables are no longer required and you can safely remove them from connect-stack.yml.

Here is a more detailed explanation of the three environment variables:

IQ_ISIM_LDAP_OFFSETS_AFTER_ISIGADI_MIGRATION

This environment variable sets the timestamp offset for ISIM LDAP source topics. The timestamp must be in the format of <isimProductName>;;;<ldapTimeStamp>. If you want to set offsets for multiple IBM Security Identity Manager products, list the multiple timestamps with a comma (,) separator. The “ldapTimeStamp” must have the format, yyyymmddhhmmssZ, where the timestamp is specified in Coordinated Universal Time.

Example 1: If you configured an IBM Security Identity Manager product as “isim6” in ISIQ, and you want to set its LDAP offset to 05/10/2020 12:00:00 in Coordinated Universal Time, set the environment variable as follows:

• IQ_ISIM_LDAP_OFFSETS_AFTER_ISIGADI_MIGRATION=isim6;;;20200510120000Z

LDAP entities created or updated after 05/10/2020 12:00:00 will be populated in ISIM source topics.

Example 2: Let’s suppose you configured two IBM Security Identity Manager products: (1) “isim6” whose LDAP offset you want to set to 04/28/2020 11:00:00 Coordinated Universal Time; and (2) “isim7” whose LDAP offset you want to set to 05/10/2020 05:30:00 Coordinated Universal Time. Specify the environment variable as follows:

• IQ_ISIM_LDAP_OFFSETS_AFTER_ISIGADI_MIGRATION=isim6;;;20200428110000Z,isim7;;; 20200510053000Z

IQ_IGISRC_USER_EVENT_ERC_OFFSET

This environment variable sets the offset for IGI’s USER_EVENT_ERC table, where account-related events are stored.

Example 1: If you configured an IGI product called “isvg1001”, and you want to process only the events with an ID greater than 4090, set the environment variable as follows:

• IQ_IGISRC_USER_EVENT_ERC_OFFSET=isvg1001;;;4090

Account events in the IGI OUT event queue with an ID of 4091 or greater will be populated in the IGI source topic.

Example 2: If you configured two IGI products, “isvg1001” and “isvg1001fp1”, with offsets 4090 and 2100, then set the environment variable as follows:

• IQ_IGISRC_USER_EVENT_ERC_OFFSET=isvg1001;;;4090,isvg1001fp1;;;2100

IQ_IGISRC_EVENT_OUT_USER_OFFSET

This environment variable set the offset for IGI’s EVENT_OUT_USER table, where user-related events are stored.

Example 1: If you configured an IGI product called “isvg1001”, and you want to process only the events with an ID greater than 1999, set the environment variable as follows:

• IQ_IGISRC_EVENT_OUT_USER_OFFSET=isvg1001;;;1999

User events in the IGI OUT event queue with an ID of 2000 or greater will be populated in the IGI source topic.

Example 2: If you configured two IGI products, “isvg1001” and “isvg1001fp1”, with offsets 1099 and 2999, then set the environment variable as follows:

• IQ_IGISRC_EVENT_OUT_USER_OFFSET=isvg1001;;;1099,isvg1001fp1;;;2999

Appendix I: Integration with Cloud Identity Analyze

ISIQ allows you to integrate IBM Security Identity Manager data to IBM Cloud Identity Analyze (CIA). By doing so, you can leverage CIA’s correlation engine to calculate Identity and Access-related risks and provide actionable insights. (Note: Since this ISIQ feature was introduced, IBM Cloud Identity Analyze has been rebranded as IBM Security Verify Analytics.)

Because CIA’s calculations rely on ISIM events (login authentications, account modifications, entitlement changes, etc.), most ISIQ accesses on behalf of CIA go against the ISIM database where event and audit information is held, and not against the LDAP directory. As a result, ISIM-to-CIA integration requires many more queries against the ISIM DB than does ISIM-to-IGI integration.

To run these queries, you must set ISIQ_ENABLE_CIA_DATA=true in connect-stack.yml. Refer to the “Tuning Recommendations” section of the ISIQ Deployment Guide for the ISIM DB indexes to create to optimize query performance. If you aren’t using ISIQ’s CIA integration feature, keep the default setting, ISIQ_ENABLE_CIA_DATA=false, to avoid unnecessary load on the database.

Configuring CIA

To configure CIA, you need to have previously installed the IBM Cloud Identity Analyze Bridge (refer to https://hub.docker.com/r/ibmcom/ciabridge) and on-boarded it in IBM Security Verify. Then, configure a CIA product in ISIQ. On the “Select a Product” page, highlight “IBM Cloud Identity Analyze” and click “Next”:

After you click the “Next” button, you must fill in the “IBM Cloud Identity Analyze” form:

ISIQ interacts with CIA’s REST server, and so most of the requested information on the form pertains to the location and credentials for connecting to that server.

Since the CIA REST server requires SSL, you must either load a CIA SSL cert into ISIQ’s truststore or enable the ISIQ_AUTOMATICALLY_IMPORT_CERTIFICATE environment variable in the connect-stack.yml file (for more information, see “Secure Communication to Applications” in the ISIQ Deployment Guide).

Note: Whenever you change a connect stack environment variable, you must (1) stop ISIQ’s app stack, (2) stop ISIQ’s connect stack, (3) restart the connect stack, and (4) restart the app stack in order to implement the change.

After you click “Configure”, wait several seconds for “cia-instance1” to be added to the list of products, and for its internal tasks to begin running.

The “Configuration created successfully” page lists the “CIASink” connector that gets started for each configured CIA instance. ISIM-CIA integration is a one-way data flow in which IBM Security Identity Manager acts as the source/producer and CIA is the sink/consumer.

To complete the ISIM-CIA integration, click “View Product” and go to the “cia-instance1” dashboard where you can subscribe to an IBM Security Identity Manager product:

After adding the subscription, and allowing a few minutes for ISIM data to be POST’d to CIA’s REST Server, click “Analyze” in your CIA dashboard. That causes the newly arrived ISIM data to be analyzed, with risks calculated for users, applications, and entitlements.

For more information about using CIA to remediate compliance issues, see the blog post, Remediate ISIM compliance issues using Cloud Identity Analyze (CIA) insights with custom actions.