Table of contents

Provisioning an instance of Master Data Connect

Before you can use the IBM® Master Data Connect service, you must provision an instance using the command line interface.

Before you begin

  • Ensure that the prerequisites are in place, the cluster has been configured, and the service has been installed.
  • Create a Secret for the passwords used by Master Data Connect internal services by running the following command:
    oc create secret generic mdc-core-init-secret --from-literal=keystore.password=<myKeystorePassword> --from-literal=cassandra.password=<myCassandraPassword> --from-literal=elasticsearch.password=<myElasticsearchPassword> --from-literal=couchdb.password=<myCouchdbPassword> --from-literal=couchdb.cookie.password=<myCouchdbPassword> -n <namespace_to_deploy_to>

    Replace the placeholder values with the appropriate passwords.

  • Get a bearer token for the user, typically a Cloud Pak for Data administrator user, who has rights to provision a new Master Data Connect instance. Run the following command:
    curl -k -X GET https://CPD-ROUTE-NAME/v1/preauth/validateAuth -H 'content-type: application/json' -H 'password: password' -H 'username: admin'
    Replace the following values:
    Variable Replace with
    CPD-ROUTE-NAME The route URL of the Cloud Pak for Data server.
    password The password of the Cloud Pak for Data administrator user.
    user The user ID of the Cloud Pak for Data administrator user.
    Take note of the bearer token returned in the response. You will need to provide it in the provisioning command.
  • Get the current version of the Master Data Connect service add-on. The add-on version number must be included as part of your provisioning request. Run the following command:
    curl -k -X POST \
      https://CPD-ROUTE-NAME/zen-data/v1/addOn/query?show_all=true \
      -H 'Authorization: Bearer BEARER-TOKEN' \
      -H 'Content-Type: application/json' \
      -d '
    {
      "type": "mdc-app"
    }'
    Replace the following values:
    Variable Replace with
    CPD-ROUTE-NAME The route URL of the Cloud Pak for Data server.
    BEARER-TOKEN The bearer token value that you got for the current user in the previous prerequisite step.
    Take note of the Version value returned in the response. You will need to provide it in the provisioning command.

Procedure

  1. Run the following command to provision a Master Data Connect instance:
    curl -k -X POST \
      https://CPD-ROUTE-NAME/zen-data/v3/service_instances \
      -H 'Authorization: Bearer BEARER-TOKEN' \
      -H 'Content-Type: application/json' \
      -d '
    {
      "addon_type": "mdc-app",
      "addon_version": "ADD-ON-VERSION",
      "create_arguments": {
        "description": "MDC-Instance",
        "parameters": {
            "docker_registry_prefix": "docker-registry.default.svc:5000/project-namespace",
            "global.docker_registry_prefix": "docker-registry.default.svc:5000/project-namespace",
            "global.persistence.storageClassName": "STORAGE-CLASS-NAME"
    
            # optional custom parameters (omit if not used)
            "OPTIONAL-CUSTOM-PARAMETER": "OPTIONAL-CUSTOM-PARAMETER-VALUE",
            "OPTIONAL-CUSTOM-PARAMETER": "OPTIONAL-CUSTOM-PARAMETER-VALUE",
            "OPTIONAL-CUSTOM-PARAMETER": "OPTIONAL-CUSTOM-PARAMETER-VALUE" 
        }
      },
      "namespace": "NAMESPACE",
      "display_name": "DISPLAY-NAME"
    }'  
    Replace the following values:
    Variable Replace with
    CPD-ROUTE-NAME The route URL of the Cloud Pak for Data server.
    BEARER-TOKEN The bearer token value that you got for the current user as part of the prerequisite steps.
    ADD-ON-VERSION The version of the Master Data Connect service add-on. Use the version value that you got for the service as part of the prerequisite steps.
    Registry prefix parameters Replace the values under the parameters section as needed. Use the values provided by your cluster administrator.
    • docker_registry_prefix - The local registry to which you transferred your images when running the cpd-linux adm command during installation.
      • The default value for OpenShift 3.11 is
        docker-registry.default.svc:5000/project-namespace
      • The default value for OpenShift 4.5 is
        image-registry.openshift-image-registry.svc:5000/project-namespace

      Replace the project-namespace variable with the namespace of your Cloud Pak for Data project.

    • global.docker_registry_prefix - The local registry to which you transferred your images when running the cpd-linux adm command during installation.
      • The default value for OpenShift 3.11 is
        docker-registry.default.svc:5000/project-namespace
      • The default value for OpenShift 4.5 is
        image-registry.openshift-image-registry.svc:5000/project-namespace

      Replace the project-namespace variable with the namespace of your Cloud Pak for Data project.

    STORAGE-CLASS-NAME The storage class name for the type of storage this Master Data Connect instance will use. For example, the storage class name may be similar to one of the following values. The exact value can differ depending on your specific client instance:
    • nfs-client (for NFS storage)
    • portworx (for Portworx)
    • ocs (for OpenShift storage)
    OPTIONAL-CUSTOM-PARAMETER Use the optional custom parameters to customize a number of operational parameters of your Master Data Connect deployment. Add your custom parameters to the parameters section of the provisioning curl request.
    OPTIONAL-CUSTOM-PARAMETER-VALUE The value that corresponds to an optional custom parameter.
    NAMESPACE The namespace being used for the Cloud Pak for Data installation. Use the value provided by your cluster administrator.
    DISPLAY-NAME Specify the display name for this instance of the Master Data Connect service.
    Note: Spaces and special characters are not allowed in the display name.
    When the command is run successfully, it will result in a 202 response that provides an ID for the new instance ID. For example:
    "id": 1582510653894
    This response triggers the creation of the instance. Instance creation takes a few minutes.
    Tip: You can monitor instance creation by running the following command:
    oc get pods |grep mdc
    The pods need to be in a Running or Completed state. Any pods that are in an Error state need to show at least one instance in Completed state (the pods time out with an error until successfully completed. For example:
    oc get pods |grep mdc
    
    mdc-addon-install-8r8gp                     0/1       Completed   0          24m
    mdc-aspera-server-0                         1/1       Running     0          1m
    mdc-cassandra-0                             1/1       Running     0          4m
    mdc-cassandra-1                             1/1       Running     0          3m
    mdc-cassandra-2                             1/1       Running     0          2m
    mdc-cassandra-enable-authentication-59rsk   0/1       Error       0          4m
    mdc-cassandra-enable-authentication-gtnx9   0/1       Error       0          4m
    mdc-cassandra-enable-authentication-ljv89   0/1       Error       0          4m
    mdc-cassandra-enable-authentication-tklq9   0/1       Completed   0          1m
    mdc-cassandra-enable-authentication-w67w6   0/1       Error       0          3m
    mdc-core-5b84d7646d-7nd56                   1/1       Running     0          1m
    mdc-core-5b84d7646d-87gwm                   1/1       Running     0          1m
    mdc-core-initialize-dp7kh                   0/1       Completed   0          5m
    mdc-core-onboarding-hlwm2                   0/1       Completed   0          1m
    mdc-core-prepare-sample-7vpsk               0/1       Completed   0          1m
    mdc-couchdb-0                               1/1       Running     0          4m
    mdc-couchdb-1                               1/1       Running     0          4m
    mdc-couchdb-2                               1/1       Running     0          4m
    mdc-couchdb-create-indexes-hfwjf            0/1       Completed   0          1m
    mdc-couchdb-initialize-cluster-7ljlb        0/1       Error       0          4m
    mdc-couchdb-initialize-cluster-b9m65        0/1       Completed   0          4m
    mdc-couchdb-initialize-cluster-qncxl        0/1       Error       0          4m
    mdc-couchdb-initialize-database-pb4zn       0/1       Completed   0          1m
    mdc-elasticsearch-data-0                    1/1       Running     0          4m
    mdc-elasticsearch-data-1                    1/1       Running     0          3m
    mdc-elasticsearch-data-2                    1/1       Running     0          2m
    mdc-elasticsearch-ingest-77c5fbb778-6ql85   1/1       Running     0          4m
    mdc-elasticsearch-ingest-77c5fbb778-hxhzp   1/1       Running     0          4m
    mdc-elasticsearch-master-0                  1/1       Running     0          4m
    mdc-elasticsearch-master-1                  1/1       Running     0          4m
    mdc-elasticsearch-master-2                  1/1       Running     0          3m
  2. When initialization is complete, verify the instance's status:
    1. Make sure that there is at least one pod showing as Completed for each of the types listed in the previous step.
    2. Ensure that all Master Data Connect pods are running.
    3. Check the service status:
      1. From the IBM Cloud Pak for Data web client, click My instances.
      2. From the Provisioned instances tab, ensure that your Master Data Connect instance appears in your list of instances.
        Note: The instance will only appear in this list if:
        • The boarding pod has completed successfully.
        • The mdc-core pod is running.
    Tip: If an instance of Master Data Connect fails to provision correctly, run the following commands to clean up any incomplete instances:
    PROJECT=$(oc status | grep "^In project" | awk '{print $3}')
    CPD_INSTALL_OPERATOR_POD=$(oc get pods --no-headers -l name=cpd-install-operator -o custom-columns=":metadata.name" | head -n 1)
    export TILLER_NAMESPACE=$PROJECT
    mkdir -p ~/.helm
    oc cp ${CPD_INSTALL_OPERATOR_POD}:/etc/certs/..data/helm.cert.pem ~/.helm/cert.pem
    oc cp ${CPD_INSTALL_OPERATOR_POD}:/etc/certs/..data/helm.key.pem ~/.helm/key.pem
    helm delete --tls RELEASE_NAME
    oc delete cm,secret,pdb,pvc,svc,pod,jobs -l app.kubernetes.io/name=Master-Data-Connect
    oc delete cm mdc-aspera-server-client-keys
    
    Replace the RELEASE_NAME value with the name of the release. The release name follows the format mdc-app-0-INSTANCE-ID. For the INSTANCE-ID parameter, use the ID provided in the response to the provisioning command.
    If the helm delete command results in a not found message, this means that the service instance was not created completely. In this case, you must manually clean all the mdc related objects that constitute a Master Data Connect service instance. To clean the Master Data Connect objects, run the following commands:
    oc get all |grep mdc
    oc delete cm,secret,pdb,pvc,state