Fronting WebSphere - Hints and Tips for Edge Caching Proxy PAC-LDAP config and replacement of TAM Plugin.
Brownrob 2700006G6F Visits (8953)
Hello, and welcome to my first blog entry on the subject of "Fronting WebSphere" with bundled Proxy and Load Balancer products like IBM HTTP Server (IHS webserver), WebSphere Plug-in proxy load balancer module for webservers, Edge Caching Proxy and Edge Load Balancers. The subject for this first blog entry is not the one I had planned but a recent support engagement resulted in some information I think is important to share about one of the more difficult configurations for Edge Caching Proxy.
Edge Caching Proxy supports using an LDAP for authorization in its support for PAC-LDAP where PAC stands for "Policy Authentication Control". The related details can be found in the Caching Proxy Administration Guide under Section: Configuring Caching Proxy security, sub section: Using the PAC-LDAP authorization module. However, configuration of Edge Caching Proxy for LDAP authorization is very rare. Edge Caching Proxy is an old product that has gone through many development support teams over the years. Caching Proxy is a "stabilized" product and therefore has minimal development support and there are no plans for a 64 bit version. It is a known bottleneck under high loads of HTTPS which is becoming ever more prominent in usage so development assumes customers will be looking for alternate general purpose proxy server products. The point being that support teams for Caching Proxy have little hands on experience in PAC-LDAP configuration for Caching Proxy.
In Caching Proxy ibmproxy.conf there are a number of directives under the section for "API Directives" to support a "custom" LDAP module for Caching Proxy. However, Caching Proxy does install a module for PAC_LDAP named "libpacwte" and directives for it are found in ibmproxy.conf under the list of "Plug-ins" for Caching Proxy, for example:
# ===== LDAP Plug-in =====
A common LDAP, like IBM Tivoli Directory Server (ITDS), provides a custom LDAP module referred to as the Tivoli Access Manager plug-in or TAM Plug-in. In ibmproxy.conf the configuration would look something like this:
#### TAM PLUGIN ####
Authorization * /opt
Usage of TAM plugin in Caching Proxy is equally as rare with minimal development support in Tivoli support. It is no surprise to Caching Proxy support, with customers scrambling to migrate to Caching Proxy V8.x to get support for TLS 1.2, to hear that TAM plugin does not work in V8.x Caching Proxy or that support has been withdrawn for TAM plugin. In 2016 Caching Proxy support helped an IBM team migrate two Edge Caching Proxy 8.5.5.x on AIX from using TAM Plugin to using PAC-LDAP plugin to authorize against an ITDS LDAP server. It still requires the ITDS LDAP client be installed on the Caching Proxy OS instance. For other LDAPs OpenLDAP client may be required. Both provide an libldap file that the Caching Proxy libpacwte module is dependent on. During the migration we discovered some interesting aspects that impeded a quick migration. This blog entry details what we learned that should help avoid some of the problems we encountered and to supplement what is in Caching Proxy Administration Guide under Configuring Caching Proxy security > Using the PAC-LDAP authorization module.
The first thing to note is that the default ibmproxy.conf for UNIX platforms had LDAP Plug-in directives (as noted above) that want to load module file libpacwte.a.so . However Caching Proxy installed two files:
Once you decide on which module file to load then run an LDD against it to check for dependent modules:
In the case that resulted in this blog entry it could not find libldap.a:
# ldd libpacwte.a
It turned out this was present in the ITDS client directories: /opt
To resolve we had to add to libpath in AIX /etc/environment file: LIBP
This resulted in a correct output of LDD:
Besides libpath problems there could be permission issues or other. The point being you may have to work with your OS admin or OS support to resolve the dependencies. The end goal in configuring the LDAP Plug-in directives for the specific libpacwte file and, resolving dependencies, so that when Caching Proxy starts there should be no message in the Caching Proxy Error log about not being able to load DLL module libpacwte. For example:
Caching Proxy Error log: Error 2 loading DLL module /opt
There is no need to proceed with configuration of the related PAC configuration files until the module can be loaded by Caching Proxy. We also re-discovered that for the LDAP Plug-in directive, Authorization http
Likewise the "Protection" configuration in ibmproxy.conf:
Protect /HOD/* Prot-LDAP
For the "Protect" directive Caching Proxy Trace shows its trying to do a URI match.
For the example: Protect /aaa/* Prot-LDAP
The directive was adjusted to allow the match:
Protect /HOD/* Prot-LDAP
The remaining configuration concerns were with the PAC conf files:
There was also a password stash file:
In the PAC conf files:
The paccp.conf is the simplest and a good one to start with. Use Caching Proxy Hostname and a free port that Caching Proxy will use to send LDAP auth request to LDAP port defined.
In this case we opted for port 2222. Netstat can be used to check for ports in use. These same values will be put in pac.conf. The remaining configuration in paccp.conf was left at default:
Pac.conf is where we start to add some information about the LDAP server. If the administrator doing the pac.ldap conf is not sure of the values they should consult their LDAP admininistrator or LDAP support. In this case the LDAP administrator gave us a good set of values to start with. We setup to test with a non-secure LDAP call to LDAP port 389. We brought over the values for paccp conf to define [PAC_MAN_SERVER]
It was for non-SSL/TLS configuration so we left the conn_type:ssl commented out. The auth
For [LDAP_SERVER], based on details from LDAP administrator we started with:
The password for admin_dn:cn=root was added to pac_ldap.cred which becomes encrypted when Caching Proxy gets restarted to test. Based on troubleshooting, a failure with the LDAP auth, we needed to use the LDAPSearch Tool, run from the Caching Proxy, to verify what configuration is needed for the pac.conf. The LDAP admin found the LDAP search tool helped to figure out PAC.conf was incorrect for:
So the final working config in PAC.conf for [LDAP_SERVER]:
During a test for PAC-LDAP it was suggested to use a policy of:
We first just added this policy to the default config in pacpolicy.conf (which had a lot of sample policies) but had problems. After further investigation we simply removed all the default policy config and left only the one policy above.
The password for admin_dn:cn=root in PAC.conf was added to pac_ldap.cred which becomes encrypted when Caching Proxy gets restarted to test.
Besides the Caching Proxy Error and Trace logs for errors specific to the PAC-LDAP there are two logs files that can be present:
Two environment variables to load noted below before starting Caching Proxy will get more debug detail related to PAC-LDAP:
Some Caching Proxy are started with a start script that is already pre-loading some ENV variables before loading ibmproxy. They could also be added on the command line before ibmproxy start.
A binary, unlimited length packet trace capture (TCPDUMP) at the Caching Proxy of the port 389/636 traffic leaving the Caching Proxy system (Destination port) is also useful.
Hopefully these notes will reduce the time to get a config for PAC-LDAP working. The primary hint/tips are:
- Resolve the module loading and dependencies first