Troubleshooting License Service Reporter

Learn how to troubleshoot License Service Reporter.

• Enabling additional information in logs for troubleshooting purposes
• No data visible in License Service Reporter after configuring data sources
• Sending historical data to License Service Reporter and resynchronizing datasources
• Troubleshooting authentication issues
  • Unable to sign in with the credentials using basic authentication
  • Unable to sign in with IBM Identity Management v4.x
• Troubleshooting authorization issues
  • Unauthorized - 403 return code
  • Error logs
  • Unable to connect to the IAM IDP issuer_url because it points to the localhost

Enabling additional information in logs for troubleshooting purposes

By default, the License Service Reporter instance pod logs contain only the basic information about the service.

You can enable additional information in logs for troubleshooting purposes by updating the IBMLicenseServiceReporter custom resource.

Complete the following steps to enable additional information in logs:

1. Log in to your OpenShift console.
2. Go to Operators > Installed Operators.
3. From the Project drop-down list, select All Projects.
4. Select IBM Licensing Operator.
5. Select the IBMLicense Service Reporter tab. The instance of License Service Reporter custom resource is listed.
6. From the overflow action menu, select IBMLicenseServiceReporter.
7. Edit the YAML. To enable additional information in logs, add the following line: logLevel: DEBUG under spec.

The available logLevel options:

• DEBUG - This option enables all debug information in logs.
• VERBOSE - This option extends the logs with information about license calculation.

Following is an example configuration:

spec:
  logLevel: DEBUG

No data visible in License Service Reporter after configuring data sources

Note: Data sent by License Service (License Service Reporter datasource) will be visible in License Service Reporter after around an hour of configuring data sources.

In case if no data is visible from datasource License Service Reporter:

• Check if you have any reported IBM software deployed on that datasource. You can check this by generating the licensing report in License Service. For more information, see Retrieving license usage of products.

• If you have the reported IBM software deployed on that data source, and still you see no data in License Service Reporter, then check the License Service and License Service Reporter logs for errors. Some common problems in the License Service logs includes:

  • Connection errors
  • Wrong License Service Reporter URL configured in License Service instance

Sending historical data to License Service Reporter and resynchronizing datasources

The described procedure covers the following scenarios:

• Resetting License Service Reporter due to the known issues or after getting instructions from the IBM support.

  Note: Following the procedure in this scenario removes all data from License Service Reporter, hence, it will completely reset the License Service Reporter. The historical data will be imported from the datasources.

• Migrating from License Advisor to License Service Reporter and transferring the historical data.

  Note:: For this scenario, start the procedure from the Step 3.

Prerequisites

• Linux or OSX console
• Access to cluster with kubectl where IBM License Service and IBM License Service Reporter is installed.
• kubectl
• jq
• gzip
• base64
• xargs

Important:

• Before you follow the procedure, generate audit snapshot from License Service Reporter (if you already deployed the License Service Reporter) and for each License Service instance. Auditing period should cover time since the first deployment for each instance.

• Additionally, note the user-defined threshold values if the value has been set. You will have to manually recover them later.

Steps

1. Backup License Service Reporter token and licensing data:

   1. Define the namespace where License Service Reporter is deployed.

      LICENSING_NAMESPACE=<enter the namespace of the License Service Reporter>
      

   2. Backup token to the file on your computer.

      kubectl get secret ibm-license-service-reporter-token -n ${LICENSING_NAMESPACE} -o=jsonpath='{.data.token}' > token.txt
      

   3. Generate an audit snapshot from License Service Reporter for the backup purpose.

2. Clean License Service Reporter database by deleting the License Service Reporter instance:

   1. Log in to your OpenShift cluster console.

   2. Go to Operators > Installed Operators.

   3. Set the project to All Projects.

   4. Find and select the IBM Licensing.

   5. In the navigation bar, go to IBM License Service Reporter tab and delete existing instance.

      

   6. Confirm that you want to delete the License Service Reporter.

   7. Wait until the License Service Reporter pod disappears.

3. Deploy the newest License Service Reporter. Wait until License Service Reporter pod is up and running.

4. Optional: Restore backed up token.

   Note: Skip this step if you deploy License Service Reporter for the first time.

   1. Apply saved value to the secret on the cluster, where the License Service reporter is deployed.

      cat token.txt | xargs -I {} kubectl patch secret ibm-license-service-reporter-token -n ${LICENSING_NAMESPACE} -p '{"data":{"token":"{}"}}'
      

   2. Restart License Service Reporter pod.

      lsrPodName=$(kubectl get pods -n ${LICENSING_NAMESPACE} | grep ibm-license-service-reporter-instance | awk '{print $1}')
      kubectl delete pod ${lsrPodName} -n ${LICENSING_NAMESPACE}
      

   3. Restart License Service pod on the same cluster where License Service Reporter is deployed.

      lsPodName=$(kubectl get pods -n ${LICENSING_NAMESPACE} | grep ibm-licensing-service-instance | awk '{print $1}')
      kubectl delete pod ${lsPodName} -n ${LICENSING_NAMESPACE}
      

5. Optional: Configure each cluster that was configured to feed data to License Advisor as a data source for License Service Reporter. For more information, see Configuring data sources for License Service Reporter.

   See the following notes:

   • Skip this step if you are resetting previously deployed License Service Reporter.

   • For the cluster where you deployed License Service Reporter, the License Service instance is automatically configured. Therefore, you do not need to perform any configuration for this cluster.

6. Synchronize License Service historical data.

   Note: This step synchronizes the license usage data only. The user-defined threshold values are not synchronized, and the values need to be set manually afterwards.

   Important: Perform this step on each cluster with License Service that is configured as License Service Reporter datasource, including License Service instance on the cluster with License Service Reporter.

   1. Upgrade License Service instance to the latest version. Wait until the pod is running.

   2. Define the synchronization start date. The start data determines the time starts when the License Service data is sent to the License Service Reporter. Provide the date in the [year,month,day] format. By default, the date is set to License Service first release date.

      START_SYNC_DATE=[2020,4,6]
      

   3. Define the namespace where License Service is deployed.

      LICENSING_NAMESPACE=ibm-licensing
      

      Note: The ibm-licensing namespace is the default namespace of the License Service. If you installed License Service in a custom namespace in your cluster, replace ibm-licensing with your custom namespace in the command.

   4. Apply the synchronization start date by running the following command:

      For OSX:

         kubectl get cm ibm-licensing-config -n ${LICENSING_NAMESPACE} -o jsonpath="{.binaryData.data}" | base64 -d | gzip -d | jq -c ".lastHubSynchronizationDate = ${START_SYNC_DATE}" | gzip | base64 | xargs -I {} kubectl patch cm ibm-licensing-config -n ${LICENSING_NAMESPACE} -p '{"binaryData":{"data":"{}"}}'
      

      For Linux:

         kubectl get cm ibm-licensing-config -n ${LICENSING_NAMESPACE} -o jsonpath="{.binaryData.data}" | base64 -d | gzip -d | jq -c ".lastHubSynchronizationDate = ${START_SYNC_DATE}" | gzip | base64 -w 0 | xargs -I {} kubectl patch cm ibm-licensing-config -n ${LICENSING_NAMESPACE} -p '{"binaryData":{"data":"{}"}}'
      

7. Wait until your data is migrated to License Service Reporter. It takes from 12 to 24 hours to migrate the data from all the clusters and show the data on the License Service Reporter UI.

8. Access the Licensing dashboard, the License Service Reporter UI. For more information, see Viewing license usage on the Licensing dashboard.

9. Recover the threshold values manually. For more information, see Setting the license usage threshold.

Troubleshooting authentication issues

You can troubleshoot the common authentication issues for License Service Reporter.

Unable to sign in with the credentials using basic authentication

You can sign in to the License Service Reporter console with the credentials retrieved from the secret. If 401: Invalid Username or Password error comes into view, complete the steps that follow:

1. Restart the operator pod with the following command. Update the namespace with the installation namespace of the License Service Reporter:

      kubectl delete pods -l app=app.kubernetes.io/instance=ibm-license-service-reporter-operator -n $NAMESPACE
   

2. Wait for sometime until the new operator pod is created.

3. Check the logs with the following command. Update the namespace with the installation namespace of the License Service Reporter:

      kubectl logs -l app.kubernetes.io/instance=ibm-license-service-reporter-operator -n $NAMESPACE
   

   The similar log entry is displayed:

      2023-06-29T11:52:01Z INFO Reconcile all done {"controller": "ibmlicenseservicereporter", "controllerGroup": "operator.ibm.com", "controllerKind": "IBMLicenseServiceReporter", "IBMLicenseServiceReporter": {"name":"instance","namespace":"ibm-licensing"}, "namespace": "ibm-licensing", "name": "instance", "reconcileID": "2f3eb70d-56c0-4042-b194-0f7408e8fd75", "resourceKind": "*v1alpha1.IBMLicenseServiceReporter", "resourceName": "instance"}
   

4. Delete the License Service Reporter pod to restart the application with the following command. Update the namespace with the installation namespace of the License Service Reporter:

     kubectl delete pods -l app=ibm-license-service-reporter-instance -n $NAMESPACE
   

5. After the application is active, you can log in with the credentials.

Unable to sign in with IBM Identity Management v4.x

IBM Identity Management does not support authorization. To sign in with IBM Identity Management, remove --allowed-group and --allowed-role parameters from the Custom Resource of the License Service Reporter.

Troubleshooting authorization issues

You can troubleshoot the common authorization issues for License Service Reporter.

Unauthorized - 403 return code

• Make sure that the proper role is assigned for the user with --allowed-group or --allowed-role parameter.
• Make sure that the user belongs to the correct email domain with --email-domain parameter.

Error logs

For more details on the authentication issues, check the logs of the authentication container in the License Service Reporter deployment.

Unable to connect to the IAM IDP issuer_url because it points to the localhost

To resolve this issue, complete the following steps:

1. For the OIDC_ISSUER_URL parameter, change the localhost to the cp-console route address in the IAM platform-auth-idp configuration map.
2. Restart the IAM platform-auth-service pod and retry with the configuration.