TRACE RESOLVER

You can activate Start of changeTrace ResolverEnd of change to collect Start of changeTrace ResolverEnd of change output. Start of changeTrace ResolverEnd of change output displays information about what the resolver looked for (the questions), where it looked (available name servers, resolver cache information, and local host table definitions), and what information was used to satisfy a resolver API request. Because the resolver runs in the address space of the application, you must collect Start of changeTrace ResolverEnd of change from the application address space. After the Start of changeTrace ResolverEnd of change output is collected, check the following information in the output:
  • Fix or check any problems reported at the top of the trace. These are errors in the resolver data sets, or they might be errors detected in the resolver setup file during initialization of the resolver address space.
  • The environment (z/OS® UNIX or native MVS™) that the application is running in determines where the resolver obtains the TCPIP.DATA statements to use. Are the data sets being used by the resolver the ones you expected? If not, see the search orders for data sets in z/OS Communications Server: IP Configuration Guide.
  • Check that the expected MVS data sets or UNIX file system files are accessible by the user or batch job. Errors detected by a security product (for example, RACF®) or OPEN services can generate messages that help indicate the problem. For example, IEC141I 013-C0 can be generated if a file does not have the correct permission bit settings to allow it to be read. RACF message ICH408I can be issued if no OMVS segment is defined or if insufficient authorization is granted to read a data set. See z/OS Communications Server: IP Configuration Guide for more information.
  • Check the TCPIP.DATA parameter values, especially NAMESERVER, NOCACHE, Start of changeNOCACHEREORDER, End of changeNSINTERADDR, NSPORTADDR and SEARCH. TCPIP.DATA parameters are explained in z/OS Communications Server: IP Configuration Reference.
  • Check the questions posed by the Resolver to DNS or in searching the local host files. Are these the queries you expected?
  • Check for the cache query and cache add attempts being attempted by the resolver as part of processing a resolver query. Are these the queries you expected, and the results that you anticipated for those requests? Are the results from the cache-query attempts returning the information that you expected to be returned for those target resources? If you suspect that there are problems with the operations of the resolver cache, you will need to collect CTRACE records for further diagnosis. See CTRACE - RESOLVER for more details.
  • Look for errors or failures in the trace.
  • Did DNS respond (if you expected it to)? If not, see whether DNS is active at the IP address you specified for NAMESERVER and NSINTERADDR and what port it is listening on. DNS logs can be helpful. Ask the DNS administrator for help.
    Tips: The resolver supports the Extension Mechanisms for DNS (EDNS0) standards, which permit DNS messages of greater than 512 bytes to be returned by DNS to the resolver; however, some network routers are configured to silently discard DNS messages of greater than 512 bytes. If the Start of changeTrace ResolverEnd of change suggests that DNS did not respond, verify that no routers between the resolver and DNS are discarding the messages.

    The resolver dynamically determines the EDNS0 capability of each DNS, based on responses or timeouts to DNS queries. The current view of the DNS capabilities is included in Start of changeTrace ResolverEnd of change output. You can issue the Netstat HOme/-h command to examine Start of changeTrace ResolverEnd of change output. Use the Start of changeTrace ResolverEnd of change information to verify that the resolver is using proper EDNS0 processing for a given DNS. Issue the MODIFY REFRESH command to force the resolver to dynamically relearn the EDNS0 capability of the name servers. Consider adjusting RESOLVERTIMEOUT values if timeout conditions cause the resolver to mistakenly avoid EDNS0 processing for a given DNS.

  • The following are some common misunderstandings:
    • If the queried name server returns NXDOMAIN, the resolver does not continue to the next name server in the list. NXDOMAIN means the domain does not exist according to that name server.
    • The resolver only appends the specific names listed in the Search (or Domain) parameter. It does not attempt shorter versions of these. For example, if you look for "johndoe" and your search list has "anywhere.usa.com", the resolver looks for "johndoe.anywhere.usa.com" and "johndoe". (The order depends on the value of option ndots.) The Resolver does not look for "johndoe.anywhere" or "johndoe.anywhere.usa" or "johndoe.usa.com" or "johndoe.com".
    • The contents of any local hosts files are not cached in the system-wide resolver cache, but are saved separately for each task.
    • Negative cache entries are created to represent the following responses from a name server:
      • A response with a return code value of NXDOMAIN. This typically represents a host name that has no records of any type in the specified domain.
      • A response with a return code value of NOERROR when no answer records are returned. This represents a host name that does not have any records of the type that was requested (A, AAAA, etc.) in the specified domain, but does have some records of a different type.
      • A response with a return code value of NOERROR when the answer records returned represent canonical, or alias, names. This represents a resource that is officially known by other names in the specified domain, does not have any records of the type that was requested (A, AAAA, etc.), but does have some records for a different type of resource.
    • Start of changeYou cannot use the cache reordering function to ensure different reordered resolution results for individual applications. The list of cached IP addresses is reordered in a round-robin manner for an individual cached host name regardless of which applications request resolution of the host name. Therefore, an individual application that issues multiple resolution requests for the same host name might receive the cached IP addresses in the same order in response to each resolution request, even if the cache reordering function is active. Resolver address sorting algorithms can also cause the same results to be generated in response to multiple requests for the same host name. See Cache reordering in z/OS Communications Server: IP Configuration Guide for more information.End of change
Activate Trace Resolver output in one of the following ways:
  • Specify the z/OS UNIX RESOLVER_TRACE environment variable or a SYSTCPT DD allocation. Specifying the RESOLVER_TRACE environment variable or allocating the SYSTCPT DDname dynamically activates Trace Resolver output regardless of the TCPIP.DATA or the _res structure resDebug specification. Dynamic activation of Trace Resolver can be useful when you are not sure where the TCPIP.DATA statements might be found.
  • Specify the TCPIP.DATA statement TRACE RESOLVER or OPTIONS DEBUG. When using a TCPIP.DATA statement to activate the trace, have the trace activation statement as your very first statement. This ensures that the trace is in effect for all statements in the TCPIP.DATA specification.
  • Set the debug option (resDebug) in an application _res structure.
  • Start of changeEnable the CTRACE TRACERES option to collect Trace Resolver output. You can use this option to debug resolver problems in one of the following conditions:
    • Long running started tasks or jobs cannot be restarted.
    • Long running started tasks or jobs have significant resolver API activity that often occurs under multiple tasks.
    You can use the CTRACE TRACERES option to collect the Trace Resolver output without restarting the started tasks or jobs. For more information, see CTRACE - RESOLVER.End of change
The resolver uses the following search order to determine whether Trace Resolver output is necessary. The Trace Resolver data is contained in the specified output location. If the output location is not available for writing, the next search location is used. The default location for the Trace Resolver output in the z/OS UNIX environment is stdout. In the native MVS environment, it is as specified by the SYSPRINT DD.
  1. The RESOLVER_TRACE environment variable (z/OS UNIX environment only).
  2. The SYSTCPT DD allocation.
  3. The TRACE RESOLVER or OPTIONS DEBUG statements. You must allocate STDOUT or SYSPRINT to generate trace data. The allocations need to exist in all operating environments including TSO, for example, your TSO Logon Procedure.
  4. The resDebug bit set to on in the _res structure option field. STDOUT or SYSPRINT must be allocated or no trace data is generated.
Trace Resolver output can be written to any of the following:
  • A TSO user terminal screen
  • z/OS UNIX STDOUT
  • JES SYSOUT
  • An MVS Sequential data set (a member of a PDS is not supported). The data set must already exist or be allocated as new with the following DCB characteristics:
    • An LRECL between 80 and 256 with a RECFM of Fixed Block.
    • For an LRECL of 128 or larger, the last six print positions are the storage address of the MVS TCB that issued the resolver call. This can be helpful with multitask applications.
  • A z/OS UNIX file system file. The file can either be an existing file or be dynamically allocated by the resolver when needed. The maximum line length used in the file is 255 characters. The last six print positions are the storage address of the MVS TCB that issued the resolver call. This can be helpful with multitask applications.

If the Trace Resolver output uses an MVS data set or z/OS UNIX file system file, the output is for the resolver services invoked by the last command or UNIX process. If possible, use SYSOUT=* or z/OS UNIX STDOUT to trace multiple resolver service invocations (for example, a multitask environment).