LDAP problem determination

In addition to the regular text messages from the HCD LDAP backend, debug output can be requested. This debug output is switched on or off by specifying a debug level for the IBM Tivoli Directory Server for z/OS (see z/OS IBM Tivoli Directory Server Administration and Use for z/OSz/OS IBM Tivoli Directory Server Administration and Use for z/OS) and is printed to the same location as the HCD LDAP backend text messages. HCD LDAP backend's debug output is only available in English language and character representation IBM-1047.

The HCD LDAP backend mainly supports two debug levels: LDAP_DEBUG_ERROR and LDAP_DEBUG_TRACE.

LDAP_DEBUG_ERROR causes all the information concerning errors, detected while performing operations, to be printed. From the HCD LDAP backend's point of view, LDAP_DEBUG_ERROR can be switched on regularly without significant performance loss.

LDAP_DEBUG_TRACE causes all important program operations to be printed. From the HCD LDAP backend's point of view, LDAP_DEBUG_TRACE should only be used when reproducing error situations for problem determination.

If you have general problems getting the IBM Tivoli Directory Server for z/OS, RACF Backend, or any other plug-ins running, leave out all HCD LDAP backend definitions from the started task and configuration files (ds.conf). If the server and plug-ins still will not run, contact the IBM service.

If everything functions correctly without the HCD LDAP backend and abends with the HCD LDAP backend, then request a dump and check there for further information.

If the dump indicates that functions in DLL GLDSLP31 or GLDCLDAP are missing, then maybe the IBM Tivoli Directory Server for z/OS functionality is not available.

HCD LDAP backend abnormal termination

There are two different locations at which the HCD LDAP backend might terminate abnormally:

• The IBM Tivoli Directory Server for z/OS address space
• The HCD instance address space

The way to find out more information about the abnormal termination depends on in which of these locations it terminated see IBM Tivoli Directory Server for z/OS address space or HCD instance address space.

In general, whenever the HCD LDAP backend or parts of it terminate abnormally, some global resources may remain in memory. These resources are so called POSIX message queues and are used for the Inter Process Communication (IPC) between the HCD LDAP backend and the HCD instances. They have to be removed from the system manually.

To remove POSIX message queues from the system proceed as follows:

1. Look into the IBM Tivoli Directory Server for z/OS started task log to find the identifiers (ID's) of all message queues allocated: The HCD LDAP backend prints out all ID's using the message CBD0004I. Note that you have to find all of these CBD0004I messages.
2. Use the UNIX System Services command line tool ipcs to list all the POSIX message queues in memory.
3. Find those identifiers in the list which correspond to POSIX message queues belonging to the abnormally terminated HCD LDAP backend.
4. Remove the appropriate message queues using UNIX System Service command line tool ipcrm.

IBM Tivoli Directory Server for z/OS address space

In general, diagnostic output can be found in the IBM Tivoli Directory Server for z/OS started task log. Investigate this log to find information, about the system or user abend code, the reason code, and additional messages and follow the instructions in HCD abnormal termination.

For more information on problem determination, refer to z/OS Problem Managementz/OS Problem Management. Read the information on how to obtain a dump of the abnormally terminated program and determine whether the problem was caused by the HCD LDAP backend or by any other program in the IBM Tivoli Directory Server for z/OS address space.

HCD instance address space

Since an HCD instance resides in a different address space (created with a call to spawn()) to that of the corresponding IBM Tivoli Directory Server for z/OS, it is not permitted to print its output to the IBM Tivoli Directory Server for z/OS started task log. Instead, an HCD instance creates an HFS file containing diagnostic output. The name and location of this file follows the conventions described in the publication z/OS Language Environment Debugging Guidez/OS Language Environment Debugging Guide in the unit "Using Language Environment Debugging Facilities". Note that the environment variable _CEE_DMPTARG is set to /tmp by default.

Example:
Assume that a language environment dump of the HCD instance with the process ID 197 is taken at 05:55:01 pm on September 18, 1998. The file containing the dump will be called CEEDUMP.19980918.175501.197 and it will be located in the HFS directory /tmp.

Adverse LDAP return code from the HCD LDAP Backend

Each LDAP client's request is answered by the IBM Tivoli Directory Server for z/OS with an LDAP response containing a return code for the request. Such a return code can be either good (LDAP_SUCCESS) or bad (e.g. LDAP_UNWILLING_TO_PERFORM). These numeric return codes can be transformed into text messages (e.g. "DSA is unwilling to perform.") using the LDAP client API.

Return codes such as LDAP_OPERATIONS_ERROR, LDAP_NO_MEMORY, or LDAP_LOCAL_ERROR typically indicate a problem within the IBM Tivoli Directory Server for z/OS, the HCD LDAP backend, or the LDAP client. If you assume the problem to be located in the IBM Tivoli Directory Server for z/OS or the HCD LDAP backend, look into the IBM Tivoli Directory Server for z/OS started task log and search for error messages.

Return codes such as LDAP_INAPPROPRIATE_MATCHING, LDAP_INVALID_DN_SYNTAX, or LDAP_UNWILLING_TO_PERFORM typically indicate an incorrect LDAP request. Check if your request conforms with the appropriate constraints. You may also check the IBM Tivoli Directory Server for z/OS started task log for error messages.

Error in the HCD LDAP backend

If the HCD LDAP backend's message CBD0009E is printed to the IBM Tivoli Directory Server for z/OS started task log, an error situation was encountered inside the HCD LDAP backend while performing an operation. Switch on IBM Tivoli Directory Server for z/OS debug level LDAP_DEBUG_ERROR and reproduce the problem. The log will contain more information concerning the reason of the error. You may additionally specify debug level LDAP_DEBUG_TRACE to get detailed informations about the operations performed by the HCD LDAP backend.

The following illustrates two typical error situations with advice on how to check for these problems.

HCD Instance Startup Fails

If an HCD instance cannot be started correctly, this will not lead to error messages in the IBM Tivoli Directory Server for z/OS started task log or to bad return codes for the LDAP client in all cases.

Consider the following possible scenario: The IBM Tivoli Directory Server for z/OS is started together with the HCD LDAP backend which, in turn, launches at least one HCD instance. All launched HCD instances fail to start. As the HCD LDAP backend has no indication that the HCD instances failed to start, it will wait forever for an acknowledgement from the HCD instances. Incoming requests will be scheduled for an HCD instance and will be blocked up until an acknowledge arrives. In this way, the requests will never be performed and no LDAP response will be sent to the requesting LDAP client.

From the LDAP client's point of view: If the very first request for the HCD LDAP backend is pending for a long time, the HCD instances might have failed to start up. In general, if one HCD instance fails to start up, all HCD instances will fail.

To check whether an HCD instance failed to start, look in the IBM Tivoli Directory Server for z/OS started task log: HCD LDAP backend's message CBD0007I will indicate how many HCD instances will be started. If all of these HCD instances are able to start up correctly, you will find an appropriate number of CBD1002I messages. If you cannot find them it's most probable that the HCD instances failed to start up.

HCD instance user ID switch fails

If you have set up the IBM Tivoli Directory Server for z/OS or the HCD LDAP backend incorrectly, the HCD instances might have problems switching their User ID appropriately. It might be possible that the User ID switch fails completely, or only for one or more specific User IDs.

If you assume that LDAP requests can't be performed because of a failed User ID switch, you can check your assumption as follows:
Search for the CBD0009E message in the IBM Tivoli Directory Server for z/OS started task log. If you can find it switch on IBM Tivoli Directory Server for z/OS debug level LDAP_DEBUG_ERROR and search for message EDC5139I. This message indicates that the User ID switch has failed.

Finally, check the setup of your HCD LDAP backend.