Troubleshooting
Problem
Network or DNS admins are reporting that an AIX system is sending excessive IPv6 DNS queries.
Resolving The Problem
1- Make sure that the name resolution order is set to suppress IPv6 lookups:
# grep -v ^# /etc/netsvc.conf
hosts = local, bind4
2- Unset NSORDER system variable if it's already set
# echo $NSORDER
local,bind
# export NSORDER=""
# echo $NSORDER
#
3- Make sure that netcd daemon is not running and configured for a resolver.
# lssrc -s netcd
When the netcd daemon is running and configured for a resolver (for example, DNS) and a map (for example, hosts), the resolution is first made using the cached answers. If it fails, the resolver is called and the response is cached by the netcd daemon.
Check to see if you have/etc/netcd.conf file configured.
# grep -v ^# /etc/netcd.conf
#
By default, everything in that file is commented out.
If the file is not configured the netcd daemon should be stopped.
# stopsrc -s netcd
Comment out the start of netcd if it is uncommented.
# vi /etc/rc.tcpip
#start /usr/sbin/netcd "$src_running"
4- Some applications may not honour the netsvc.conf /etc/netsvc.conf name resolution order and use their own. To know which application is sending those DNS queries you would need to follow this procedure:
- Start an iptrace which looks only at port 53 for DNS
# startsrc -s iptrace -a "-a -p 53 -b /tmp/iptrace.trc"
- Start a kernel trace
# trace -a -j 101,104,106,107,10C,134,139,252,255,465,536,539 -T 80000000 -L 160000000 -o /tmp/ktrace.bin ; sleep 10 ; trcstop
- Stop the iptrace
# stopsrc -s iptrace
- Collect process table output
# ps -ef > /tmp/ps-ef.out
- Format the kernel trace
# trcrpt -O pid=on,exec=on /tmp/ktrace.bin > /tmp/ktrace.out
- Use Wireshark or ipreport to find the IPv6 dns requests (type=AAAA) note the local port number and convert it to hex
i.e port=44360 --> 0xAD48
Example of how to convert the above port to HEX
# bc -l
obase=16 <enter>
44360 <enter>
AD48
<ctrl D> to exit
*NOTE: Wireshark can be downloaded online and is free. If you decide to go with ipreport, then use this command to convert from binary to text format:
# ipreport -rns /tmp/iptrace.bin > /tmp/iptrace.txt
- Search in the kernel trace using this local port number (lport) number obtained above (e.g. AD48)
# vi /tmp/iptrace.txt
539 dcm 2289732 51912861 0.000860537 0.001744 PCB return from in_pcbhashlookup2 _faddr=F1000200067B981A fport=0000000000000035_laddr=F1000200067B981E lport=000000000000AD48
there you can see the process and parent ID's.
- Now you can search the ps -ef output (saved earlier) for these process ID's to get the corresponding process name and thus find the problem application.
# vi /tmp/ps-ef.out
Was this topic helpful?
Document Information
Modified date:
17 June 2018
UID
isg3T1026923