The AIX Detective: "Don't reboot"
AnthonyEnglish 270000RKFN Visits (8507)
Caller: "Hello, is that the AIX detective? I've got a problem on my LPAR. I can't add a new LUN. Everyone wants me to reboot, but I thought I'd give you a call first."
AIX emergency hotline: "You made the right decision. My years of experience, and my gut instinct tell me,
I could have ignored it, but the new LUNs weren't recognised on the LPAR. Clearly this was going to require some serious detective work.
Detective step 1: who's making all the fuss?
It was the command cfg_perfvmmstat which was reporting a problem. What on earth is that? Performance? vmstat?
The AIX file command showed me the type of file it is.
# file /usr
An executable. I might have known. Where does this mysterious command
Detective step 2: who put him up to it?
I found out what fileset the principal suspect belonged to, so I could establish a crime scene and
Detective step 3: establish a crime scene
Were there any filesets in a damaged state? Maybe the last AIX install was interrupted.
No output from that command. Fine. No filesets in a broken state.
Detective step 4: being on the level
Time to check the surrounds for other clues. What OS were they running?
5300-11-01-0944AIX 5.3 TL 11. That's pretty recent. I also saw other LPARs running the same level without any cfgmgr problems. What gives?
Detective step 5: what time did the crime occur?
09:22AM up 324 days, 9:40, 1 user, load average: 0.42, 0.43, 0.31
324 days since the last reboot? That seems a long time. Just curious, what day is it today?
Wait a minute!
Julian date (what day of the year is this?)
# date "+%j"How many days before that did the reboot happen?
So 71 days before the end of 2009 was the last time this LPAR was rebooted.
71 days is about 10 weeks, or week 42 of 2009. But the oslevel showed me:
5300-11-01-0944the fix pack from AIX was only dated from week 44 of 2009. So, either
Sometimes, as a detective you've just got to go with your instincts. I tossed a coin and went for option 2.
The new LUNs came up and there were no errorrs reported from cfgmgr.
The AIX detective went home satisfied. Another crime solved.