IBM Support

Unable to create Sandbox on IBM Cloud

Troubleshooting


Problem

Create a Sandbox on IBM Cloud by using a WaziSandboxSystem resource with an empty PersistentVolumeClaim fails to create the sandbox instance.

Symptom

The sandbox never becomes available.

Cause

The WaziSandboxSystem does not set the correct permissions for the init container. Therefore, the creation process cannot download the volumes to the PersistentVolumeClaim and exits with a failure.

OpenShift sees the failure and retries. However, since the deployment is always the same, the init container never succeeds, and the sandbox never starts.

Environment

IBM Wazi Developer for Red Hat CodeReady Workspaces 1.1.0 on IBM Cloud.

Diagnosing The Problem

  1. Use the OpenShift web console, and find the WaziSandboxSystem Details page for the sandbox.
  2. Select the Resources tab.
  3. The Resources tab shows a list with the Deployment and Service. Click the link for the Deployment to go to the Deployment Details page.
  4. If the Conditions section of the deployment for Type Available includes a status of False with the Reason: MinimumReplicasUnavailable, continue with the next step. Otherwise, the problem is not the same as the one that is described in this note.
  5. Select the Pods tab.
  6. If the list of pods shows that the pod has a status of Init:CrashLoopBackOff and the readiness of ContainersNotInitialized, continue with the next step. Otherwise, the problem is not the same as the one described in this note.
  7. Click the link for the pod Name to go to the Pod Details page.
  8. Select the Logs tab.
  9. Choose the log for wazi-sandbox-system-copy in the drop-down. Check the following error at the beginning of the log.
    SEVERE: Failed with an exception:4: 
     4: 
        at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:1003)
        at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:888)
        at
            com.ibm.zdt.zvolumes.GetZVolumesHandler.run(GetZVolumesHandler.java:91)
     Caused by:
            java.io.FileNotFoundException:
     /zdt/zvolumes/inventory.txt (Permission denied)
    

    If this error is found, proceed with the Resolving the Problem section. Otherwise, the problem is not the same as the one that is described in this note.

Resolving The Problem

To solve the problem, complete the following steps.
  • Preparation
    1. Save the WaziSandboxSystem resource YAML to a file sandbox.yaml.
    2. Delete the WaziSandboxSystem resource that fails by following the instructions that are described in the Deleting a sandbox instance.
  • Create a ConfigMap if required.
    If the sandbox.yaml contains a dynamic configuration for the license server and SFTP server, you need to create a ConfigMap that contains this information. The dynamic configuration is being used if the spec.configMap.create value is true, and there will be other keys in the spec.configMap section. For example,
    apiVersion: sandbox.wazi.ibm.com/v1alpha1
    kind: WaziSandboxSystem
    metadata:
      name: your-sandbox-name
    spec:
      zpdt:
        secretName: your-sandbox-secret
        configMap:
          create: true
          licenseServer: your-license-server
          sftpHost: your-sftp-server
          sftpPort: 22
          sftpUser: sftp-user
          sftpPath: /path/to/adcd
      zVolumes:
        existingClaimName: your-sandbox-storage
    

    If the dynamic configuration is not being used, skip to Create a PersistentVolumeClaim if required. Otherwise, use the following steps to create a ConfigMap.

    1. Create a file configmap.yaml by using the following template.
      apiVersion: v1
      kind: ConfigMap
      metadata:
        name: sandbox-fix-init-config
      data:
        licenseServer: ls.example.com
        sftpHost: sftp.example.com
        sftpPort: ‘22’
        sftpUser: example-user
        sftpPath: /path/to/adcd
      
    2. Change the values in the configmap.yaml to match the values in the spec.configMap from the sandbox.yaml. If the sftpPort does not appear in the sandbox.yaml, you can leave the value as 22.
    3. Save the configmap.yaml.
    4. Submit the configmap.yaml to your namespace by using the OpenShift web console or by using the following command.
      oc create -f configmap.yaml
    5. Edit the sandbox.yaml.
    6. In the spec.configMap section, delete all existing keys.
    7. In the spec.configMap section, add the key and value: name: sandbox-fix-init-config.
      Note: If you changed the metadata.name field in the template, use the new name here as well.
    8. Save the sandbox.yaml.
  • Create a PersistentVolumeClaime if required.
    If the sandbox.yaml uses a dynamic storage, you need to create a ConfigMap that contains this information. The dynamic storage is being used if the spec.zVolumes.existingClaimName key does not exist, or the value is empty, or the string is empty, and there will be other keys in the spec.zVolumes section. For example:
    apiVersion: sandbox.wazi.ibm.com/v1alpha1
    kind: WaziSandboxSystem
    metadata:
      name: your-sandbox-name
    spec:
      zVolumes:
        size: 300Gi
        storageClassName: ""
      zpdt:
        secretName: your-sandbox-secret
        configMap:
          name: your-sandbox-config
    

    If a dynamic storage is not being used, spec.zVolumes.existingClaimName will be set, skip to Create a WaziSandboxVolumeCopy request. Otherwise, use the following steps to create a PersistentVolumeClaim.

    1. Create a file pvc.yaml by using the following template.
      kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: sandbox-fix-init-storage
      spec:
        storageClassName: VALUE_FROM_SANDBOX_YAML
        resources:
          requests:
            storage: SIZE_FROM_SANDBOX_YAML
        selector:
          matchExpressions:
            - key: ZVOLUMES_SELECTOR_LABEL_FROM_SANDBOX_YAML
              operator: In
              values: [ ZVOLUMES_SELECTOR_VALUE_FROM_SANDBOX_YAML
        accessModes:
          - ReadWriteOnce
        volumeMode: Filesystem
      
    2. Change the VALUE_FROM_SANDBOX_YAML to the spec.zVolumes.storageClassName value from the sandbox.yaml. Or, change it to be “” if no such value exits in the sandbox.yaml.
    3. Change the SIZE_FROM_SANDBOX_YAML to the spec.zVolumes.size value from the sandbox.yaml.
    4. If the sandbox.yaml includes a spec.zVolumes.selector section, change the ZVOLUMES_SELECTOR_LABEL_FROM_SANDBOX_YAML to the spec.zVolumes.selector.label value and to the spec.zVolumes.selector.value value from the sandbox.yaml. Otherwise, delete the spec.selector section from the pvc.yaml.
    5. Save the pvc.yaml.
    6. Submit the pvc.yaml to your namespace by using the OpenShift web console or by using the following command.
      oc create -f pvc.yaml
    7. Edit the sandbox.yaml.
    8. In the spec.zVolumes section, delete all existing keys.
    9. In the spec.zVolumes section, add the key and value: existingClaimName: sandbox-fix-init-storage.
      Note: If you changed the metadata.name field in the template, use the new name here as well.
    10. Save the sandbox.yaml.
  • Create a WaziSandboxVolumeCopy request.
    1. Create a file volcopy.yaml by using the following template.
      apiVersion: sandbox.wazi.ibm.com/v1alpha1
      kind: WaziSandboxVolumeCopy
      metadata:
        name: sandbox-fix-init-copy
      spec:
        source:
          type: sftp
          existingConfigName: CONFIGMAP_NAME_FROM_SANDBOX_YAML
          secretName: SECRETNAME_FROM_SANDBOX_YAML
        destination:
          existingClaimName: ZVOLUMES_EXISTINGCLAIMNAME_FROM_SANDBOX_YAML
        imagePullSecrets:
          - name: PULL_SECRET_NAME_FROM_SANDBOX_YAML
      
    2. Change the CONFIGMAP_NAME_FROM_SANDBOX_YAML to spec.zpdt.configMap.name value from the sandbox.yaml.
    3. Change the SECRETNAME_FROM_SANDBOX_YAML to spec.zpdt.secretName value from the sandbox.yaml.
    4. Change the ZVOLUMES_EXISTINGCLAIMNAME_FROM_SANDBOX_YAML to spec.zVolumes.existingClaimName value from the sandbox.yaml.
    5. If the sandbox.yaml includes an imagePullSecrets section, change the PULL_SECRET_NAME_FROM_SANDBOX_YAML to the value from the sandbox.yaml. Otherwise, delete the imagePullSecrets section.
    6. Save the volcopy.yaml.
    7. Submit the volcopy.yaml to your namespace by using the OpenShift web console or by using the following command.
      oc create -f volcopy.yaml
    8. Wait for the Job that is created by this request to complete. Verify that the job is completed successfully by drilling down from the Job to the Pod. Check the status on the Overview tab and log file on the Logs tab to verify that the job was able to correctly copy all z/OS volume files.
  • Submit the sandbox.yaml.
    Then, you can resubmit the sandbox.yaml file, which might be modified during the earlier steps either from the OpenShift web console or by using the following command.
    oc create -f sandbox.yaml

Document Location

Worldwide

[{"Line of Business":{"code":"LOB35","label":"Mainframe SW"},"Business Unit":{"code":"BU058","label":"IBM Infrastructure w\/TPS"},"Product":{"code":"SSSQT8","label":"IBM Wazi for Red Hat CodeReady Workspaces"},"ARM Category":[{"code":"a8m0z000000cxz2AAA","label":"Wazi Developer for Red Hat CodeReady->Sandbox"}],"ARM Case Number":"","Platform":[{"code":"PF025","label":"Platform Independent"}],"Version":"1.1.0"}]

Document Information

Modified date:
26 October 2021

UID

ibm16375272