I am having a problem with our CSM management server following an upgrade from CSM 1.7.0
I upgraded our CSM management server to 22.214.171.124 and afterwards, dsh no longer works, and
therefore none of the csm commands are working (updatenode, cfmupdatenode for example).
We have /usr/bin/ssh set as our remote shell for under csmconfig.
Our management server is running RedHat ES 4 update 4 (32bit)
Client nodes run RHEL4u4 (32-bit), RHEL5.2(64-bit) and RHEL5.4(64-bit) with most
The problems seems to be limited to nodes running RHEL4.4, which would include our CSM
I have nodes running RHEL 5.2 (64bit), and after manually upgrading the all the csm and rsct
packages on the client node (running 5.2), I am able to dsh commands from that client to the
management server and also to other clients without problem.
ex: on the upgraded client (RHEL 5.2 64-bit), I run the following command:
/opt/csm/bin/dsh -r /usr/bin/ssh -n <node> date
and the date command is executed.
on the management server, the same command just hangs, with no errors. If I Ctrl-C the command,
I get the date output along with the error that remote shell had an exit code of 255. If I
give a command that has no STDOUT, it seems works just fine
I was able to recreate the problem on one of our client nodes running RHEL4.4.
The node had the CSM 1.7.0 installed, and the above command worked just fine
to any other node (upgraded or not).
Next, I updated the csm packages to csm126.96.36.199, and the node is no longer able to execute
the dsh command. I even upgraded the openssh packages to no avail.
This seems to be a problem with the RHEL4.4 support and ssh. I could solve part of the problem
by reloading the management server with RHEL5.2, but I want to do this only as a last resort.
I would greatly appreciate any help on this problem.
Pinned topic dsh problems after csm upgrade to 188.8.131.52
Answered question This question has been answered.
Unanswered question This question has not been answered yet.
Updated on 2010-02-01T06:21:29Z at 2010-02-01T06:21:29Z by SystemAdmin
bejohn2 2700025DKW2 Posts
Re: dsh problems after csm upgrade to 184.108.40.206010-01-15T18:35:29ZThis is the accepted answer. This is the accepted answer.I uninstalled csm.dsh-220.127.116.11-27 on the client node and reinstalled csm.dsh-18.104.22.168-21, and the client is once again able to dsh commands to other nodes.
I did the same on the management server, along with csm.server-22.214.171.124-27 to csm.server-126.96.36.199-21, and it now seems to be functioning properly again.
SystemAdmin 110000D4XK476 Posts
Re: dsh problems after csm upgrade to 188.8.131.52010-01-30T18:02:30ZThis is the accepted answer. This is the accepted answer.I'm getting a similar problem - running RHEL4.8 on the management server, 5.4 on most of the clients, and CSM 184.108.40.206 on the entire cluster. It seems like if anything goes to stderr, then dsh hangs. If I send a command to multiple nodes, all it takes is some stderr output on one of them, and then all the responses get blocked until I Ctrl-C.
At the same time this started happening, I also started seeing rsync hangs when doing a cfmupdatenode command. It doesn't always hang but still enough to be an annoyance. We have a cron job that does a cfmupdatenode command every hour to all 400+ nodes in the cluster. After a day, I'll often see several hundred stuck rsync processes in the management server.
I don't have the option of backing off to an earlier version of csm, as earlier versions don't support RHEL5.4. I'm hoping that upgrading the management server to RHEL5.4 will help but it will be a while before i can schedule that.
SystemAdmin 110000D4XK476 Posts
Re: dsh problems after csm upgrade to 220.127.116.11010-02-01T06:21:29ZThis is the accepted answer. This is the accepted answer.
- SystemAdmin 110000D4XK