Topic
  • 8 replies
  • Latest Post - ‏2015-12-14T14:25:48Z by swlinn
SriHarshaVardhan
SriHarshaVardhan
7 Posts

Pinned topic Load Balancer Status

‏2013-06-23T09:00:41Z |

I have a Load balancer object  with 3 members (WAS Servers). I enabled health check option in lb with default settings.  Even though the members are up.lb status option showing all the members as down. And unable to process any transaction coming to the proxy.

could any one help me in understanding how to configure health check for lb. Do i need to change health check port & any other customization is required.

 

  • Rohit-Goyal
    Rohit-Goyal
    151 Posts

    Re: Load Balancer Status

    ‏2013-06-23T09:48:07Z  

    You have to make sure that backends responding for health check calls coming from DataPower. 

    Rohit

  • swlinn
    swlinn
    1395 Posts

    Re: Load Balancer Status

    ‏2013-06-26T02:35:38Z  

    Assuming you're using a "Standard" type health check, the health ports on the member are used, and if not specified, the health port on the health page is then used.  You have two options, one SOAP which will send a SOAP request to your backend, the other a HTTP GET request.  In either case, you should have an XPath to validate if you get a response.  A / will be sufficient to just say you got an XML response.  As Rohit indicates, your backend url that you specify will get this request, and must respond with an XML response.

    Regards,

    Steve

  • SriHarshaVardhan
    SriHarshaVardhan
    7 Posts

    Re: Load Balancer Status

    ‏2013-07-03T18:12:05Z  
    • swlinn
    • ‏2013-06-26T02:35:38Z

    Assuming you're using a "Standard" type health check, the health ports on the member are used, and if not specified, the health port on the health page is then used.  You have two options, one SOAP which will send a SOAP request to your backend, the other a HTTP GET request.  In either case, you should have an XPath to validate if you get a response.  A / will be sufficient to just say you got an XML response.  As Rohit indicates, your backend url that you specify will get this request, and must respond with an XML response.

    Regards,

    Steve

    Hi Steve,

    This is my observation :

    Case 1: Disabled Health Check  toggle in health check tab,.

                   We make one was instance down (say for A instance)

                   Statrted hitting proxy ; we noticed that A instance went into soft down status and requests are 

    routed to b,c instances. we are good with this behaviour.

     

    Case 2: Enables health check toggle in healthcheck tab.

    with in 2 minutes are servers went down even though all the servers are up.

    we are using default healthcheck settings. even health check port also we did not change. No heath check port is configured in members tab..its blank.

     

    please let me know do we need any customization to settings when we enable health check.?

    even though i disabled health check ; servers are going to soft down state when ever servers are down..this behaviour is fuling my requirement. tahnk what is the need of enabling health check ?

     

     

     

  • swlinn
    swlinn
    1395 Posts

    Re: Load Balancer Status

    ‏2013-07-09T16:59:43Z  

    Hi Steve,

    This is my observation :

    Case 1: Disabled Health Check  toggle in health check tab,.

                   We make one was instance down (say for A instance)

                   Statrted hitting proxy ; we noticed that A instance went into soft down status and requests are 

    routed to b,c instances. we are good with this behaviour.

     

    Case 2: Enables health check toggle in healthcheck tab.

    with in 2 minutes are servers went down even though all the servers are up.

    we are using default healthcheck settings. even health check port also we did not change. No heath check port is configured in members tab..its blank.

     

    please let me know do we need any customization to settings when we enable health check.?

    even though i disabled health check ; servers are going to soft down state when ever servers are down..this behaviour is fuling my requirement. tahnk what is the need of enabling health check ?

     

     

     

    Sorry for the lag in responding as I was on vacation last week and am catching up. 

    Case 1.  Since the one member is really down, the LBG will attempt to connect to it, but when it fails, it goes into a "quarantined" state as specified by the softdown state.  The LBG will not attempt this member again until the damp interval, by default 2 minutes, has expired.  You may see a delay every two minutes when that one member is retried, but after failing again it will go back into softdown state.

    Case 2.  So the health check is failing which is why the members go into a down state.  Perhaps you could provide an export of your LBG object as determining why the health checks are failing will depend upon your configuration and what your backend supports.  You mention you don't have a health port, so the port on the health page will be used for health check requests.  Is this port active on these servers to respond with an appropriate XML response?  The health check type, whether soap requests are to be sent (vs a HTTP GET) and the specific URI of that request will determine what health check is sent, and the xpath will determine if the XML response is accepted as valid.

    The reason for health checks is to not attempt to send traffic to backend members that are down to avoid any latency caused by waiting for the connection to fail.  It is a more proactive step to periodically determine if the backend is accepting requests.  Also, the health check frequency is generally less than the two minute damp time, so if the server comes back up you'll have that server back in service quicker.

    Regards,

    Steve

  • mtz
    mtz
    45 Posts

    Re: Load Balancer Status

    ‏2015-01-06T06:39:12Z  
    • swlinn
    • ‏2013-07-09T16:59:43Z

    Sorry for the lag in responding as I was on vacation last week and am catching up. 

    Case 1.  Since the one member is really down, the LBG will attempt to connect to it, but when it fails, it goes into a "quarantined" state as specified by the softdown state.  The LBG will not attempt this member again until the damp interval, by default 2 minutes, has expired.  You may see a delay every two minutes when that one member is retried, but after failing again it will go back into softdown state.

    Case 2.  So the health check is failing which is why the members go into a down state.  Perhaps you could provide an export of your LBG object as determining why the health checks are failing will depend upon your configuration and what your backend supports.  You mention you don't have a health port, so the port on the health page will be used for health check requests.  Is this port active on these servers to respond with an appropriate XML response?  The health check type, whether soap requests are to be sent (vs a HTTP GET) and the specific URI of that request will determine what health check is sent, and the xpath will determine if the XML response is accepted as valid.

    The reason for health checks is to not attempt to send traffic to backend members that are down to avoid any latency caused by waiting for the connection to fail.  It is a more proactive step to periodically determine if the backend is accepting requests.  Also, the health check frequency is generally less than the two minute damp time, so if the server comes back up you'll have that server back in service quicker.

    Regards,

    Steve

    Hi Steve,

    Apologies for replying on a very old thread but had to clear a doubt. Why is default value of health check frequency i.e. 180 seconds more than damp time default value i.e. 120 secs?

    Doesn't that mean that a faulty server will be put back into rotation after damp time is over even though health check hasn't been performed for that server yet? It means that for upto 60 seconds there might be failures if the requests are routed to that server which is still not healthy.

     

    Thanks,

    Mohit

  • soaDevArch
    soaDevArch
    80 Posts

    Re: Load Balancer Status

    ‏2015-12-12T00:07:11Z  

    We know with HC tab ( healtch and custom hc) would work for backend which can send xml response. How about if it is something which supports json ( or any non-xml ) based backend systems.

     

    Regards,

    Salla

  • kenhygh
    kenhygh
    1916 Posts

    Re: Load Balancer Status

    ‏2015-12-12T12:55:25Z  

    We know with HC tab ( healtch and custom hc) would work for backend which can send xml response. How about if it is something which supports json ( or any non-xml ) based backend systems.

     

    Regards,

    Salla

    Salla,

    There have been improvements in this area in 7.1 and 7.2, check the release notes.

  • swlinn
    swlinn
    1395 Posts

    Re: Load Balancer Status

    ‏2015-12-14T14:25:48Z  
    • mtz
    • ‏2015-01-06T06:39:12Z

    Hi Steve,

    Apologies for replying on a very old thread but had to clear a doubt. Why is default value of health check frequency i.e. 180 seconds more than damp time default value i.e. 120 secs?

    Doesn't that mean that a faulty server will be put back into rotation after damp time is over even though health check hasn't been performed for that server yet? It means that for upto 60 seconds there might be failures if the requests are routed to that server which is still not healthy.

     

    Thanks,

    Mohit

    Hi Mohit, 

    My apologies for not responding sooner.  Health check frequencies and the damp time are to separate subjects.

     

    I can't speak to why these default values were set when load balancer groups were added to the firmware a long time ago, but your statement "60 seconds of failures" is not correct.  When a transaction attempts to use a load balancer group, a connection attempt will be made to that member.  If a connection cannot be made, the member is placed into "softdown" state and a connection attempt to the next member based on the load balancer group algorithm will be done.  Once is "softdown" state, the member will not be used by the distribution algorithm, and it will be brought back into an "up" state after the damp time has expired.  Once in the "up" state, if the connection to be backend still cannot be done, it will again be placed into "softdown" state for another damp time duration.

     

    Health checks are sent to each member based on this frequency.  If the connection can be made and the health check response is validated, then the member will be set to an "up" state, otherwise to a "down" state.  Again only "up" members are attempted unless you have the property to try every member before failing.  I believe health checks are done independently and regardless of the member state, so if you happened to have a member in "softdown" state and the health check succeeded, it would be placed into an "up" state and not have to wait for the remaining damp time.

     

    Hi Salla,

    some specific 7.2 improvements that you should be aware of:

    1. health check response can now be non-XML and can be validated using gatewayscript code instead of an xsl

    2. health check requests can also use a variety of HTTP methods beyond GET and POST

    3. the ldap and ims connect methods are tcp level health checks, but their use is very restrictive because the configuration of the health port was very specific to each type.  Ldap used the health port only on the health tab, meaning the assumption was that your members had to be on the same port which isn't necessarily the case.  IMS used the server port on the member page to be the health port, but previous releases had issues in the retry frequency of the health check should the health check failed.  Both of these still exist in 7.2 but have been deprecated.  In their place is a new type "TCP Connection" which continues to do the tcp level health checks, but the health port specification is consistent with the standard type, namely, the health port of the member if specified, and if not specified there, on the health tab.  Finally, you how have two types of TCP connections, Full which establishes a TCP connection, or Partial which accepts the server as available if a SYN ACK is received but does NOT establish a TCP connection.

     

    Regards,
    Steve