Topic
  • 7 replies
  • Latest Post - ‏2009-12-07T04:20:50Z by SystemAdmin
SystemAdmin
SystemAdmin
476 Posts

Pinned topic Recycled IP addresses.

‏2009-11-23T02:09:14Z |
We just installed a new bladecenter, and I'm installing Linux on the new nodes. Three of the 14 installs failed, with a message like:

makenode: The /opt/csm/bin/mgmtsvr -k -n xxxx.yyy.com 10.10.10.10 command failed with a return code of 1 .
makenode: An error occurred while attempting to execute the mgmtsvr command.
mgmtsvr: Exit code 1 from command: /usr/bin/mkrsrc-api IBM.ManagementServer::Hostname::"10.10.10.10"::ManagerType::"CSM"::LocalHostNamesList::'{"xxxx.yyy.com"}'::MgmtSvrHostNamesList::'{"10.10.10.10"}'::LocalHostname::"xxxx.yyy.com" 2>&1

I found that the one thing the three failing nodes have in common is that they use the same IP addresses that were used by former nodes in the cluster. Those nodes were deleted long ago with the rmnode command and their entries were removed from the DNS, but there are still entries for them in /csminstall/csm/config and in /etc/dhcpd.conf. Is there something that can be done to make csm completely forget them? And is this likely to be the cause of these errors?

tia
Updated on 2009-12-07T04:20:50Z at 2009-12-07T04:20:50Z by SystemAdmin
  • SystemAdmin
    SystemAdmin
    476 Posts

    Re: Recycled IP addresses.

    ‏2009-11-23T06:53:19Z  
    The rmnode should remove the entries in /csminstall/csm/config and /etc/dhcpd.conf. I'm not sure why there were still entries for your old nodes. Maybe you can remove the entries manually and try to updatenode again. And also make sure the /etc/hosts does not contain the entries for your old nodes.
  • SystemAdmin
    SystemAdmin
    476 Posts

    Re: Recycled IP addresses.

    ‏2009-11-24T13:54:20Z  
    These nodes are not in /etc/hosts. I've removed every file under /csminstall/csm that has the offending hostnames as part of their names. But they're still showing up in dhcpd.conf after a csmsetupinstall command. Are there other good places to look for them?
  • SystemAdmin
    SystemAdmin
    476 Posts

    Re: Recycled IP addresses.

    ‏2009-11-25T03:37:28Z  
    These nodes are not in /etc/hosts. I've removed every file under /csminstall/csm that has the offending hostnames as part of their names. But they're still showing up in dhcpd.conf after a csmsetupinstall command. Are there other good places to look for them?
    Could you make sure that the nodes you want to install can be resolved to the desired IPs? For example, you can ping the nodename and make sure it targets to the correct IP and the old IPs never show up.
  • megalosaurus
    megalosaurus
    3 Posts

    Re: Recycled IP addresses.

    ‏2009-12-01T15:40:03Z  
    The new nodes are properly defined in both the nameserver and in /etc/hosts. The old nodes are not defined anywhere that I'm aware of. but somehow they are getting defined in dhcpd.conf after a csmsetupinstall command.
  • SystemAdmin
    SystemAdmin
    476 Posts

    Re: Recycled IP addresses.

    ‏2009-12-05T12:36:26Z  
    I finally managed to clear these entries from dhcpd.conf. I had to put the files back into /csminstall/csm/config and then use the makedhcp command to remove them from dhcpd.conf. Then I went through csmsetupinstall and installnode and they didn't reappear in dhcpd.conf. But the install still failed with the same error as before. As far as I can tell, the only thing these nodes have in common is their recycled IP addresses.
  • SystemAdmin
    SystemAdmin
    476 Posts

    Re: Recycled IP addresses.

    ‏2009-12-07T00:59:03Z  
    More on this problem... I tried doing lsrsrc on every defined resource class. I found the IP addresses of the three offending nodes associated with their old hostnames in IBM.MNNetworkInterface, IBM.MngNode, and IBM.MngNodeNetIF. For lack of any better leads, I have to assume this is the leading suspect. But I don't know what's the best way to clear them out of there. I don't want to do it the wrong way and make things worse.
  • SystemAdmin
    SystemAdmin
    476 Posts

    Re: Recycled IP addresses.

    ‏2009-12-07T04:20:50Z  
    More on this problem... I tried doing lsrsrc on every defined resource class. I found the IP addresses of the three offending nodes associated with their old hostnames in IBM.MNNetworkInterface, IBM.MngNode, and IBM.MngNodeNetIF. For lack of any better leads, I have to assume this is the leading suspect. But I don't know what's the best way to clear them out of there. I don't want to do it the wrong way and make things worse.
    Can you try to use rmrsrc to remove them?