PowerVM/VIOS How to fix "The RMC has a status code of busy. One reason for this can be the incorrect sizing of the VIOS."

Steps

The following steps are only to be followed after checking;

• VIOS resources (Virtual I/O Server Performance Advisor)
• HMC RMC (Checking the status of RMC connections)
• PowerVC resources (Planning for performance)
• VIOS CMDB refresh;

# /usr/bin/stopsrc -s vio_daemon
#lssrc -a |grep vio_daemon --> make sure it is inoperative
# /usr/sbin/slibclean
# rm -rf /home/ios/CM
# /usr/bin/startsrc -s vio_daemon -a '-d 4'
#lssrc -a |grep vio_daemon --> make sure it is Active and note down the PID
Then we need to execute a kill -1 vio_daemon's PID to start the populate of the DB

• If the issue persists after checking the above, then we need to reset and cleanup the CM, the following steps do not require downtime but it is recommended to run them during a quiet window, as there might be a brief temporary loss of manageability, also please make sure other VIO servers are up and running while working on this VIOS;

• 1. Prepare log directory

rm -R /tmp/CMreset 2>/dev/null

mkdir /tmp/CMreset

echo "Saving logs in: /tmp/CMreset/CMcleanup.history.log" >> /tmp/CMreset/CMcleanup.history.log

• 2. Log hostname and current CM processes

echo "====== locate hostname ======" >> /tmp/CMreset/CMcleanup.history.log

host `hostname` >> /tmp/CMreset/CMcleanup.history.log

echo "======" >> /tmp/CMreset/CMcleanup.history.log

date >> /tmp/CMreset/CMcleanup.history.log

echo "====== ps -ef of CM before cleanup starting ======" >> /tmp/CMreset/CMcleanup.history.log

ps -ef | grep CM >> /tmp/CMreset/CMcleanup.history.log

• 3. Restart inetd

stopsrc -s inetd

startsrc -s inetd

sleep 10

• 4. Stop vio_daemon

stopsrc -s vio_daemon

• 5. Force kill vio_daemon if still running

pid=$(ps -ef | grep -v grep | grep vio_daemon | awk '{print $2}')

if [ -n "$pid" ]; then

    kill -9 $pid

fi

• 6. Remove CMDB directories

rm -R /var/vio/CM 2>/dev/null

rm -R /home/ios/CM 2>/dev/null

• 7. Clear library cache

slibclean

• 8. Restart vio_daemon

startsrc -s vio_daemon

sleep 60

• 9. Log post-cleanup info

echo "======" >> /tmp/CMreset/CMcleanup.history.log

date >> /tmp/CMreset/CMcleanup.history.log

echo "======  ps -ef of CM after cleanup completed ======" >> /tmp/CMreset/CMcleanup.history.log

ps -ef | grep CM >> /tmp/CMreset/CMcleanup.history.log

echo "====== listing files in /home/ios/CM/DB after cleanup ======" >> /tmp/CMreset/CMcleanup.history.log

ls -ltrb /home/ios/CM/DB >> /tmp/CMreset/CMcleanup.history.log

sleep 180

echo "====== listing files in /home/ios/CM/DB after 180 seconds ======" >> /tmp/CMreset/CMcleanup.history.log

ls -ltrb /home/ios/CM/DB >> /tmp/CMreset/CMcleanup.history.log

echo "====== Last few lines from viod_CM.log after cleanup ======" >> /tmp/CMreset/CMcleanup.history.log

tail -n 60 /home/ios/logs/viod_CM.log >> /tmp/CMreset/CMcleanup.history.log

• 10. Signal vio_daemon to reload

pid=$(ps -ef | grep -v grep | grep vio_daemon | awk '{print $2}')

if [ -n "$pid" ]; then

    kill -1 $pid

fi