Check out my latest article on Power IT Pro, "Compare Files Easily with the comm Command": http://poweritpro.com/system-admin/compare-files-easily-comm-command
Brian Smith's AIX / UNIX / Linux / Open Source blog
Version 1.2 of EZH (Easy HMC Command Line Interface) has been released.
EZH is a script for the IBM HMC console to provide an alternate, easier to use, command line interface for many common commands and the goal of the project is to make the HMC command line interface easier to use for day to day administration tasks. It also includes an optional interactive menu to make it even easier to use.
EZH is Open Source and 100% free, and is very easy to install. For more information and to download, see the project page at http://ezh.sourceforge.net/
Last month an article I wrote on Tracing IBM AIX hdisks back to IBM System Storage SAN Volume Controller (SVC) volumes was published on IBM developerWorks.
The article included a script designed to automate the process of tracing back AIX hdisks back to SVC Volumes so that you could easily see all the SVC related information about the AIX hdisk, including the SVC volume name.
Dan Aldridge modified the script in the article with a few very nice improvements. It no longer depends on pcmpath, supports more versions of the SVC, and even works with SVC Volumes presented through Virtual SCSI (* Requires AIX 6.1 TL7 or later). For more information on what the script does and how to use it, see the original article linked above.
Here is the modified script from Dan:
Check out my latest article on Power IT Pro, "Boost Your Productivity with Single-Line AIX Shell Scripts": http://poweritpro.com/aix/boost-your-productivity-single-line-aix-shell-scripts
Being careful not to shoot yourself in the foot when cleaning up users and home directories on AIX and Linux
It is possible on Linux or AIX to have users that share a home directory. For example, you might have user1, user2, and user3 all have their home directories set to /sharedhome.
Anytime you are deleting users and home directories you need to keep this in mind. You might only need to delete "user1" but want to leave "user2" and "user3" unaffected. But if you delete user1 and its home directory you might end up deleting the shared home directory which would have a big impact on "user2" and "user3".
Let's look at this situation:
AIX and Red Hat both have a "userdel" command that will optionally erase the users home directory as well ("-r" flag).
On most versions of AIX, if you did a "userdel -r" on any one of the 3 users, it would deleted the /sharedhome directory which would impact the remaining 2 users that you didn't want to affect.
On Red Hat Linux, "userdel" tries to be smarter, and verifies that the owner of the home directory matches the user that is being deleted. Thus, if you did a "userdel -r user1" it would happily wipe out /sharedhome, but if you did a "userdel -r user2" or "userdel -r user3" it wouldn't delete /sharedhome because the owner of the directory doesn't match the user being deleted.
Here is a one-liner that will do a better job checking to see if a given directory is a shared home directory between multiple users:
Change "/sharedhome" to whatever directory you would like to check. If this comes back and says it is a shared home directory, then you need to do more research before attempting to delete the home directory.
So anytime you are cleaning up users and home directories, always keep in mind that users might have shared home directories and that under some circumstances AIX and Linux will wipe out these shared home directories which might affect other users on the system.
I've noticed that on some AIX servers there are these files:
These files are there on some servers, but not on others, so I was curious what their purpose was and what created them.
I started by googling the file names, and found a few references to them, but nothing that really explained why they are there or where they come from. So then I tried searching through all man pages with a command like this:
Based on this, it sounded to be like the files were a automatically generated backup file of these files.. The server I was working on didn't have a "/etc/opasswd", or "/etc/ogroup" files, so I tried creating a user, changing a users password, and deleting a user. After all this, the "/etc/opasswd" and "/etc/ogroup" files still didn't exist as I had expected them to....
Next, I decided to look through all the executables on the server and see which of them contained a reference to "/etc/opasswd":
# for dir in `echo $PATH | tr ":" " "`; do for file in `ls -1 $dir 2>/dev/null`; do [ -x $dir/$file ] && [ -f $dir/$file ] && strings $dir/$file | grep -q -i '/etc/opasswd' && echo "$file"; done; done
The only two executables found on the server with references to /etc/opasswd where "pwdck" and "usrck". I also ran "strings" on the libraries under /usr/lib but couldn't find any references to opasswd in them.
So I did some more experimenting and found that the "pwdck" and "usrck" commands do in fact make backup copies of files to /etc/opasswd, /etc/ogroup, /etc/security/opasswd, and /etc/security/ouser. These backup files could be used to help recover the system if pwdck or usrck messed something up on the system.. Mystery solved :)
Over the years I've noticed that a lot of the core utilities on AIX are actually shell scripts.
Here are some examples of these utilities on AIX that are either shell scripts (ksh/csh) or in some cases Perl scripts:
As you can see, there are some pretty important commands in this list. And this is just a small sample of them. On my AIX server I found that there are over 400 scripts included as part of base AIX! You can see a full list of all the scripts that make up your system by running a command like this:
It is pretty cool that so many of the core commands/utilities on AIX are made up of shell scripts. For one, it shows that shell scripts can take on very important and critical tasks. It can also be extremely helpful to be able to review the scripts if you are having any issues with any of these commands. And these scripts can be an excellent learning tool. These are extremely well written and robust scripts many of which have been used for decades on thousands and thousands of servers.
Installp is the main software packaging format used on AIX. When you install a installp fileset on an AIX server it installs files in to the appropriate directories on the AIX server. You can easily see a list of files that a installp package installed on the server by running "lslpp -f fileset" (where fileset is the name of any installed fileset).
When you install a installp fileset there is more going on then just files getting copied to the right place. Most filesets also include several scripts that are run when the software is installed or uninstalled. These scripts can do almost anything from stopping/starting processes, creating files, deleting files, changing configuration files or system settings, etc.
If you are having problems installing, upgrading, or uninstalling a installp fileset you might need to take a look at the filesets installp related scripts to troubleshoot the issue.
It is easy to extract these scripts following these steps. In this example, we are working with the SDDPCM fileset.
1. List the files in the fileset and look for the liblpp.a files:
The ./usr/lpp/devices.sddpcm.61/inst_root/liblpp.a file is for the root part of the install, and the ./usr/lpp/devices.sddpcm.61/liblpp.a is for the usr part of the install.
2. Extract the liblpp.a files:
The files are extracted relative to whatever directory you are currently in.
3. Extract the liblpp.a files:
The liblpp.a files contain the scripts, and can be extracted with the "ar" command:
The scripts have now all been extracted. The "config" and "unconfig" scripts are run at install time and uninstall time, respectively. There are also several other scripts that come in to play and a explaination of what each script does is available at: http://publib.boulder.ibm.com/infocenter/aix/v6r1/index.jsp?topic=%2Fcom.ibm.aix.genprogc%2Fdoc%2Fgenprogc%2Fpkging_sw4_install.htm
A little known feature in SMIT when selecting software to install is the ability to scroll over to the right using the right arrow key to view the actual fileset name rather than just the fileset description. This makes it easier to verify that you are installing the desired fileset.
I had been using AIX for many years before I found this feature. I think it is not very commonly known because even if you have a large terminal window the software selection screen is always the same small size and doesn't ever show this information unless you scroll over with the right arrow key. The message says "Use arrow keys to scroll" but I had always thought you could only scroll up/down in the list.
Here is an animated GIF image that visually shows what I'm talking about (the image is large so it might take a minute to download in your web browser):
Over the years I've come across software that during the installation checks the AIX verison/oslevel before installation. If it detects a AIX oslevel it doesn't recognize, a lot of software will flat out refuse to install. For example, if the software was intended to be run on AIX 5.3 and you are trying to install it on AIX 6.1 the installation program might detect this and refuse to install. Some software might even check for a specific technology level and refuse to install on other TL versions.
If the software installation program is a shell script, the easiest and best way to force an install is to look through the installation shell script and find where it is checking the oslevel. Then either modify or comment out the applicable parts of the shell script to allow it to install on your version of AIX. However, if the installation program is a binary program this won't be a possibility. You can use the "file" command to check if a program is a shell script or a binary executable.
Another option is to try temporarily changing the AIX oslevel command. If the installation program is calling the "oslevel" command to determine what AIX version you are on, you can fake it out by creating your own custom "oslevel" command temporarily. To do this, first rename the /usr/bin/oslevel command to something like "oslevel_original". Then create a new /usr/bin/oslevel shell script file. The shell script will just consist of something like this:
Basically you've just created a fake, hard coded oslevel command that you can have report any OS level that the software installation is expecting. When you run oslevel now it will return whatever you hard coded in to the script. Now you do NOT want to leave your system in this state permanently - this would just be a temporary change during the software installation and then you would want to restore the original oslevel command as soon as possible.
The software might also be calling the "uname -v" and/or "uname -r" commands, and you could also create a temporary shell script to replace uname to temporarily fake out this command as well. For example, if you wanted uname to appear to return the results from an AIX 5.2 server you could use this shell script to replace the uname program:
When you run this modified uname script with the "-v" flag it would display 5, and when you run it with a "-r" flag it would display 2 (i.e. AIX 5.2). Again, this would only be a temporary change you would want to make for the software installation and then you would want to restore the original uname command.
These methods will allow you to fake out the AIX version for the vast majority of software installation programs, however some applications might use different methods to determine the oslevel such as making system calls. In these cases there might not be an easy way to fake the AIX version. If you are having a hard time determining how the installation program is determining the AIX level you could try to run the installation program with "truss" to see what system calls it is making.
Like I mentioned earlier there are a variety of situations you might be in where you need to install software on a version of AIX that the software doesn't support. However, always keep in mind that this would not be supported by the software vendor and the software might not work or have other unexpected results. Also, always remember to restore your original oslevel and/or uname commands if you hard code them to return other information.
brian_s 270002K5X3 8,817 Views
I had a new article published today on IBM developerWorks: Tracing IBM AIX hdisks back to IBM System Storage SAN Volume Controller (SVC) volumes
This is an update to my previous post on a Script to show recent Error Report (errpt) entries on AIX
Anthony and Dan had some good suggestions such as being able to specify the interval to go back in days instead of just minutes, and also having an option to just have the script show error report entries since the last time the script was run.
So below is version two of this script.
The changes are:
Here is the updated script:
Update 10/24/13: See also Version 2 of script to show recent Error Report entries on AIX
Here is a script that will show you recent Error Report (errpt) entries on AIX. As an argument to the script you specify the number of minutes you want to go back, and the script will only show errpt entries that have occurred within that many minutes from now.
This can be helpful as a standalone utility, or as part of a monitoring script that would automatically notify you if a new errpt entry came up within the last few minutes.
For example, to only show error's that have occurred within the last 15 minutes:
Or the last hour:
Or the last day:
Here is a screenshot:
Every Filesystem in AIX has two sets of permissions: The permissions on the mount point directory, and the permissions on the mounted filesystem.
Here is an example:
Normally the Mount Point Permissions don't come in to play once the filesystem is mounted (however here is an post that shows what I recommend for them)
However, if a user doesn't have read/execute permissions on the mount point you will see weird behavior and frequently have application issues as well.
Here is an example showing this:
As a non-root user, we do an "ls -al" in the directory and get a weird "./..: Permission denied" error. This is caused because the underlying mount point permissions are restricted (700) and the user doesn't have read/execute permissions on the underlying mount point (even though the mounted filesystem has 777 permissions).
Now there are 2 different ways to check to see what the permissions are on the underlying mount points of your filesystems. You can unmount the filesystem, and do an "ls -ald" on the mount point (but it will probably require application downtime to unmount the filesystem...) Or you could use this handy script that will show you the underlying mount point permissions while the filesystem is online and mounted.
Just a quick disclaimer however... These scripts have worked with my limited testing; but use them at your own risk. The IBM documentation always recommends unmounting the filesystem to check or change mount point permissions and this is the safest and best way to do it. These scripts will do everything with the filesystem mounted and online.
When run, it will show you the underlying mount point permissions for all mounted JFS/JFS2 filesystems:
If you have filesystems with too restrictive mount points that are causing you issues (like /test1 and /app2 in the example above), then you can either unmount the filesystem and change the mount point permissions, or use this script to add read/execute permissions to user/group/others on the underlying mount point directory while it is still mounted and online:
With the script you specify a filesystem and it will add the read/execute permissions on the underlying mount point:
One of the fundamental principles of troubleshooting any issue is to look for what has changed between the time things went from working to not working. One thing that could be relevant to any issue is if any software has been recently updated or installed.
AIX provides the "lslpp -h" command which will show fileset installation and update dates for each fileset. Unfortunately it doesn't sort the output of this, so it can be very difficult to look through the output to find filesets that have been recently updated or installed.
Here is a one liner that will sort the output and show you the most recently installed and updated filesets on your AIX system:
If you pipe it to "tail" you can see the 10 most recently installed or updated filesets on your AIX system:
Note: This one liner is assuming the date output will be MM/DD/YY... If you are in a different locale you might need to modify the one liner a little bit or temporarily change the environment variables to set your system to output dates in MM/DD/YY.
Also, in case you are wondering about the "sed 's/70/-70'" and "sed 's/-70/70'" parts of the script.. I noticed that lslpp -h on some servers will list the dates for some filesets as the year 70 (as in the UNIX epoch). Since "70" (1970) is bigger than "13" (2013) these old "1970" filesets were getting listed as the newest. So before the "sort" of the output I search and replace any 70's with -70's so they will be sorted correctly, and then after the sort change all the -70's back to 70. This way the dates are sorted correctly and the output still looks good.