Unable to access the portal sites

If you are unable to access the login page of a portal site, getting a blank page, an HTTP 404, or some other error.

A portal site outage can occur for one of the following reasons:
  • The portal site is down.
  • The portal subsystem is down (and so all sites are down).
  • The access attempt is not reaching the portal due to a problem in the network or Kubernetes environment.
Note: If you are on VMware, some of the following checks involve running kubectl commands on your VMs. To access your VMs for running kubectl commands:
  1. SSH to one of your subsystem VMs as the user 'apicadm', for example: ssh apicadm@mgmt1.example.com
  2. Sudo to the root user: sudo -i
  3. From here you can run kubectl commands, for example:
    # kubectl get pods
    NAME                                                              READY   STATUS      RESTARTS      AGE
    abc-management-analytics-proxy-5f8bcd74c4-jbhch                   1/1     Running     1 (97d ago)   113d
    ...
  4. API Connect pods run in the default namespace on VMware, so you do not need to specify the namespace with -n in the kubectl commands.
Check for the following problems:
  • Does the access attempt reach the portal subsystem?
    To confirm that your browser is able to reach the portal site, check the admin container logs of the portal's www pods. The nginx process that runs in the admin container is the entry point for site access. For every attempt to access a portal site, the admin container logs the HTTP call, for example:
    # kubectl logs abc-portal-site1-www-0 -c admin | grep nginx
    ...
    [     nginx stdout]   584 ce6ce0:adad58:00580c 2022-05-27 12:57:38: "9.145.11.161, 9.20.85.206" portal.site1.example.com "GET /providerorg1/sandbox/ HTTP/1.1" 200 5026 5315 5992 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.64 Safari/537.36" 0.330 "-"
    ...
    Note: On v10.0.6, v10.0.5.4 and previous releases, the nginx logs are in the web container. Use the same command, but replace -c admin with -c web.
    If the admin container logs nothing at the time the site access is attempted, then check for:
    • Network problems between your browser and the portal subsystem.
    • Problems with Kubernetes ingresses or OpenShift routes to the portal web endpoint.
    • If you are on VMware, then the tcpdump utility is installed, and you can use it to confirm if traffic is reaching your portal VM. For example:
      tcpdump -i any src host 9.20.86.22 and dst port 443
  • Are the portal subsystem pods all up and running?

    Run kubectl get ptl to check that your portal pods are running. If this does not show that all pods are running, then check which pods are failing: kubectl get pods. If you are on VMware, then gather postmortem logs and open a support case. If you are on Kubernetes or OpenShift, then check why the pods are failing to start with kubectl describe <pod name>. On the portal subsystem, the -db- pods must be running first; the other pods do not start until the -db- pods are running.

  • If the admin container logs show that the access attempt is received, then check the web container logs to see whether there are any errors at the PHP level.