Provisioning an instance of Master Data Connect

Before you can use the IBM® Master Data Connect service, you must provision an instance using the command line interface.

Before you begin

Ensure that the prerequisites are in place, the cluster has been configured, and the service has been installed.

Create a Secret for the passwords used by Master Data Connect internal services by running the following command:

oc create secret generic mdc-core-init-secret --from-literal=keystore.password=<myKeystorePassword> --from-literal=cassandra.password=<myCassandraPassword> --from-literal=elasticsearch.password=<myElasticsearchPassword> --from-literal=couchdb.password=<myCouchdbPassword> --from-literal=couchdb.cookie.password=<myCouchdbPassword> -n <namespace_to_deploy_to>

Replace the placeholder values with the appropriate passwords.

Get a bearer token for the user, typically a Cloud Pak for Data administrator user, who has rights to provision a new Master Data Connect instance. Run the following command:

curl -k -X GET https://CPD-ROUTE-NAME/v1/preauth/validateAuth -H 'content-type: application/json' -H 'password: password' -H 'username: admin'

Replace the following values:

Variable	Replace with
`CPD-ROUTE-NAME`	The route URL of the Cloud Pak for Data server.
`password`	The password of the Cloud Pak for Data administrator user.
`user`	The user ID of the Cloud Pak for Data administrator user.

Take note of the bearer token returned in the response. You will need to provide it in the provisioning command.

Get the current version of the Master Data Connect service add-on. The add-on version number must be included as part of your provisioning request. Run the following command:

curl -k -X POST \
  https://CPD-ROUTE-NAME/zen-data/v1/addOn/query?show_all=true \
  -H 'Authorization: Bearer BEARER-TOKEN' \
  -H 'Content-Type: application/json' \
  -d '
{
  "type": "mdc-app"
}'

Replace the following values:

Variable	Replace with
`CPD-ROUTE-NAME`	The route URL of the Cloud Pak for Data server.
`BEARER-TOKEN`	The bearer token value that you got for the current user in the previous prerequisite step.

Take note of the Version value returned in the response. You will need to provide it in the provisioning command.

Procedure

Run the following command to provision a Master Data Connect instance:

curl -k -X POST \
  https://CPD-ROUTE-NAME/zen-data/v3/service_instances \
  -H 'Authorization: Bearer BEARER-TOKEN' \
  -H 'Content-Type: application/json' \
  -d '
{
  "addon_type": "mdc-app",
  "addon_version": "ADD-ON-VERSION",
  "create_arguments": {
    "description": "MDC-Instance",
    "parameters": {
        "docker_registry_prefix": "docker-registry.default.svc:5000/project-namespace",
        "global.docker_registry_prefix": "docker-registry.default.svc:5000/project-namespace",
        "global.persistence.storageClassName": "STORAGE-CLASS-NAME"

        # optional custom parameters (omit if not used)
        "OPTIONAL-CUSTOM-PARAMETER": "OPTIONAL-CUSTOM-PARAMETER-VALUE",
        "OPTIONAL-CUSTOM-PARAMETER": "OPTIONAL-CUSTOM-PARAMETER-VALUE",
        "OPTIONAL-CUSTOM-PARAMETER": "OPTIONAL-CUSTOM-PARAMETER-VALUE" 
    }
  },
  "namespace": "NAMESPACE",
  "display_name": "DISPLAY-NAME"
}'

Replace the following values:

Variable	Replace with
`CPD-ROUTE-NAME`	The route URL of the Cloud Pak for Data server.
`BEARER-TOKEN`	The bearer token value that you got for the current user as part of the prerequisite steps.
`ADD-ON-VERSION`	The version of the Master Data Connect service add-on. Use the version value that you got for the service as part of the prerequisite steps.
Registry prefix parameters	Replace the values under the `parameters` section as needed. Use the values provided by your cluster administrator. `docker_registry_prefix` - The local registry to which you transferred your images when running the `cpd-linux adm` command during installation. The default value for OpenShift 3.11 is `docker-registry.default.svc:5000/project-namespace` The default value for OpenShift 4.5 is `image-registry.openshift-image-registry.svc:5000/project-namespace` Replace the `project-namespace` variable with the namespace of your Cloud Pak for Data project. `global.docker_registry_prefix` - The local registry to which you transferred your images when running the `cpd-linux adm` command during installation. The default value for OpenShift 3.11 is `docker-registry.default.svc:5000/project-namespace` The default value for OpenShift 4.5 is `image-registry.openshift-image-registry.svc:5000/project-namespace` Replace the `project-namespace` variable with the namespace of your Cloud Pak for Data project.
`STORAGE-CLASS-NAME`	The storage class name for the type of storage this Master Data Connect instance will use. For example, the storage class name may be similar to one of the following values. The exact value can differ depending on your specific client instance: `nfs-client` (for NFS storage) `portworx` (for Portworx) `ocs` (for OpenShift storage)
`OPTIONAL-CUSTOM-PARAMETER`	Use the optional custom parameters to customize a number of operational parameters of your Master Data Connect deployment. Add your custom parameters to the `parameters` section of the provisioning `curl` request.
`OPTIONAL-CUSTOM-PARAMETER-VALUE`	The value that corresponds to an optional custom parameter.
`NAMESPACE`	The namespace being used for the Cloud Pak for Data installation. Use the value provided by your cluster administrator.
`DISPLAY-NAME`	Specify the display name for this instance of the Master Data Connect service. Note: Spaces and special characters are not allowed in the display name.

When the command is run successfully, it will result in a 202 response that provides an ID for the new instance ID. For example:

"id": 1582510653894

This response triggers the creation of the instance. Instance creation takes a few minutes.

Tip: You can monitor instance creation by running the following command:

oc get pods |grep mdc

The pods need to be in a Running or Completed state. Any pods that are in an Error state need to show at least one instance in Completed state (the pods time out with an error until successfully completed. For example:

oc get pods |grep mdc

mdc-addon-install-8r8gp                     0/1       Completed   0          24m
mdc-aspera-server-0                         1/1       Running     0          1m
mdc-cassandra-0                             1/1       Running     0          4m
mdc-cassandra-1                             1/1       Running     0          3m
mdc-cassandra-2                             1/1       Running     0          2m
mdc-cassandra-enable-authentication-59rsk   0/1       Error       0          4m
mdc-cassandra-enable-authentication-gtnx9   0/1       Error       0          4m
mdc-cassandra-enable-authentication-ljv89   0/1       Error       0          4m
mdc-cassandra-enable-authentication-tklq9   0/1       Completed   0          1m
mdc-cassandra-enable-authentication-w67w6   0/1       Error       0          3m
mdc-core-5b84d7646d-7nd56                   1/1       Running     0          1m
mdc-core-5b84d7646d-87gwm                   1/1       Running     0          1m
mdc-core-initialize-dp7kh                   0/1       Completed   0          5m
mdc-core-onboarding-hlwm2                   0/1       Completed   0          1m
mdc-core-prepare-sample-7vpsk               0/1       Completed   0          1m
mdc-couchdb-0                               1/1       Running     0          4m
mdc-couchdb-1                               1/1       Running     0          4m
mdc-couchdb-2                               1/1       Running     0          4m
mdc-couchdb-create-indexes-hfwjf            0/1       Completed   0          1m
mdc-couchdb-initialize-cluster-7ljlb        0/1       Error       0          4m
mdc-couchdb-initialize-cluster-b9m65        0/1       Completed   0          4m
mdc-couchdb-initialize-cluster-qncxl        0/1       Error       0          4m
mdc-couchdb-initialize-database-pb4zn       0/1       Completed   0          1m
mdc-elasticsearch-data-0                    1/1       Running     0          4m
mdc-elasticsearch-data-1                    1/1       Running     0          3m
mdc-elasticsearch-data-2                    1/1       Running     0          2m
mdc-elasticsearch-ingest-77c5fbb778-6ql85   1/1       Running     0          4m
mdc-elasticsearch-ingest-77c5fbb778-hxhzp   1/1       Running     0          4m
mdc-elasticsearch-master-0                  1/1       Running     0          4m
mdc-elasticsearch-master-1                  1/1       Running     0          4m
mdc-elasticsearch-master-2                  1/1       Running     0          3m

When initialization is complete, verify the instance's status:
1. Make sure that there is at least one pod showing as Completed for each of the types listed in the previous step.
2. Ensure that all Master Data Connect pods are running.
3. Check the service status:
  1. From the IBM Cloud Pak for Data web client, click My instances.
  2. From the Provisioned instances tab, ensure that your Master Data Connect instance appears in your list of instances.
    Note: The instance will only appear in this list if:
    
    The boarding pod has completed successfully.
    
    The mdc-core pod is running.
Tip: If an instance of Master Data Connect fails to provision correctly, run the following commands to clean up any incomplete instances:
```
PROJECT=$(oc status | grep "^In project" | awk '{print $3}')
CPD_INSTALL_OPERATOR_POD=$(oc get pods --no-headers -l name=cpd-install-operator -o custom-columns=":metadata.name" | head -n 1)
export TILLER_NAMESPACE=$PROJECT
mkdir -p ~/.helm
oc cp ${CPD_INSTALL_OPERATOR_POD}:/etc/certs/..data/helm.cert.pem ~/.helm/cert.pem
oc cp ${CPD_INSTALL_OPERATOR_POD}:/etc/certs/..data/helm.key.pem ~/.helm/key.pem
helm delete --tls RELEASE_NAME
oc delete cm,secret,pdb,pvc,svc,pod,jobs -l app.kubernetes.io/name=Master-Data-Connect
oc delete cm mdc-aspera-server-client-keys
```
Replace the RELEASE_NAME value with the name of the release. The release name follows the format mdc-app-0-INSTANCE-ID. For the INSTANCE-ID parameter, use the ID provided in the response to the provisioning command.
If the helm delete command results in a not found message, this means that the service instance was not created completely. In this case, you must manually clean all the mdc related objects that constitute a Master Data Connect service instance. To clean the Master Data Connect objects, run the following commands:
```
oc get all |grep mdc
oc delete cm,secret,pdb,pvc,state
```

Table of contents

Provisioning an instance of Master Data Connect

Before you begin

Procedure