Tiller cannot communicate with Helm-API when on different master nodes

Tiller cannot communicate with helm-api and Rudder in a high availability environment when they are on different master nodes.

Symptoms

After installation of your high availability system, there are communication problems between Tiller and helm-api and Rudder.

Causes

The port that is used by the master nodes to communicate is blocked by a firewall.

Diagnosing the problem

Check the containers log for the following error messages:

  E0312 15:50:28.486733 1 helm.go:250] Failed to connect to Tiller: context deadline exceeded
  E0312 15:50:28.486786 1 server.go:219] failed to connect to tiller: context deadline exceeded

Resolving the issue

Enable the port for the master nodes to communicate by completing the following steps on each master node.

1. Run the following command to open the port:

   firewall-cmd --permanent --add-port=44134/tcp
   

2. Run the following command to reload the firewall settings:

   firewall-cmd --reload
   

3. Restart the master node and wait for all of the pods to show a running state.

4. Repeat those steps for the other master nodes in your environment.