Brian Smith's AIX / UNIX / Linux / Open Source blog
|Modified on by brian_s|
The Shell History file has saved my bacon many, many times. So many times I have used it to figure out what happened with a server after some kind of problem, or to find a command I had run before that I couldn't remember. The Shell history paints a very good picture of what has occurred on a server and is one of the first places I start when troubleshooting a server to see what has recently been happening.
It can also help document the configuration on a server. I remember many years ago we were having issues with a VIO SEA adapter and had to recreate it. Luckily we found the original command that was used to create the SEA in the history file so it was a piece of cake for us to recreate it.
By default the AIX history has a couple of limitations:
Many of these low limits we see today are archaic leftovers from a time long ago when computers had extremely limited resources. For example, one thing that drives me crazy is the default scroll back buffer of the popular PuTTY SSH client is only 200 lines! In an era of desktops with at least several Gigabytes of memory we certainly have room to store more than 200 measly lines of text! Luckily it is very easy to change the default scroll back buffer in PuTTY to something more reasonable like 10 million lines..
So in today's modern AIX environments where servers are often connected to terabytes of storage and often have hundreds of Gigabytes of RAM I think we have room to store a little more than 500 lines of shell history as well.
By default the AIX shell history file doesn't record the date/time stamp for each command that is run. This reduces the usefulness of the history file because the commands in it could have been run 10 minutes ago or 10 months ago, you just can't tell one way or the other. When troubleshooting an issue it can be extremely useful to be able to see that a command in the shell history was run at the same time a problem began.
Luckily it is extremely easy to fix both of these limitations with the default AIX Shell History. Simply add these lines to your global /etc/profile file:
These will increase the history size to 10,000 lines and record date/time stamps for each command run. Use the "-t" flag with the history command to see the date/time stamps next to each command.
Hopefully these simple tips will allow you to get more out of the AIX Shell History.
Being careful not to shoot yourself in the foot when cleaning up users and home directories on AIX and Linux
It is possible on Linux or AIX to have users that share a home directory. For example, you might have user1, user2, and user3 all have their home directories set to /sharedhome.
Anytime you are deleting users and home directories you need to keep this in mind. You might only need to delete "user1" but want to leave "user2" and "user3" unaffected. But if you delete user1 and its home directory you might end up deleting the shared home directory which would have a big impact on "user2" and "user3".
Let's look at this situation:
AIX and Red Hat both have a "userdel" command that will optionally erase the users home directory as well ("-r" flag).
On most versions of AIX, if you did a "userdel -r" on any one of the 3 users, it would deleted the /sharedhome directory which would impact the remaining 2 users that you didn't want to affect.
On Red Hat Linux, "userdel" tries to be smarter, and verifies that the owner of the home directory matches the user that is being deleted. Thus, if you did a "userdel -r user1" it would happily wipe out /sharedhome, but if you did a "userdel -r user2" or "userdel -r user3" it wouldn't delete /sharedhome because the owner of the directory doesn't match the user being deleted.
Here is a one-liner that will do a better job checking to see if a given directory is a shared home directory between multiple users:
Change "/sharedhome" to whatever directory you would like to check. If this comes back and says it is a shared home directory, then you need to do more research before attempting to delete the home directory.
So anytime you are cleaning up users and home directories, always keep in mind that users might have shared home directories and that under some circumstances AIX and Linux will wipe out these shared home directories which might affect other users on the system.
This is an update to my previous post on Visualizing the Physical Layout of an AIX Volume Group (see the previous post for full details). Sebastian posted a comment suggesting an option to consolidate/merge PP's that are sequentially numbered to make the script usable on hdisks that might have over 50,000 PP's. This is a great idea, and I was able to update the script to add this.
The options for the script have changed:
-v <vgname> (Specifies VG name, required)
-c ## (Optional - specifies number of columns, defaults to 2 in merge mode, 3 in non-merge mode)
-m (Optionally Enables merge/consolidate mode for sequentially numbered PP's. In parentheses it shows number of PP's and size of merged chunk. )
Here is a screenshot of the new "merge mode". If you leave off the "-m" it will show details on all PP's (see my previous post for screenshots of that mode).
Here is the updated script:
9/23/13 Update - See this updated verison of the script as well.
Here is a script I've written to visualize the physical layout of an AIX volume group. The script visually shows the location of every Physical Partition (PP) on each hdisk (AKA Physical Volume). The output shows which Logical Volume (LV) is on each of the PP's (or if it is free space). The output is color coded so each LV has its own color so that it is very easy to see where each LV physically is across the entire Volume Group. You can specify the number of columns of output depending on the size of your screen.
The intended use of the script is to show a visual representation of the Volume Group to make using commands which move around LP's/PP's such as migratelp easier to use, to make LVM/disk maintenance easier, and also as a learning tool.
Here are a few screenshots:
When running the script you specify 2 parameters: The volume group name, and the number of columns you would like displayed (or it will default to 3 columns if not specified).
Here is the script:
One common problem I have personally made and seen others make while shell scripting is trying to set a variable to be the contents of a file or the output of a multiline command, and then trying to echo the variable to process it further with grep, awk, a while loop, etc. Depending on how you do this you might get some unexpected results because you might have missing newlines and spaces from your output.
In this example, we have a file named "testfile" that contains 5 lines. I then set the testvar variable to contain the contents of the line by running "testvar=`cat testfile`". However, when I attempt to "echo $testvar" all the lines of the file are shown on one line! If I try to grep for "test line 2" it still see's everything on one line.
The same problem happens when trying to set testvar to be the output of "ls -al" and then echoing the variable:
So what is going on here? It all has to do with how command line parameters are parsed by the shell and provided to the echo command.
Here is an example that illustrates what is happening:
Notice how the echo command removed all the extra spaces out of what I had typed? It is doing this because the shell is parsing each word as an argument and providing the arguments to the echo command:
The shell is parsing arguments and providing them to the echo command. As part of parsing the arguments, all of the extra spaces are being removed by the shell before they are passed to the echo command. This is the same thing that was happening when our newlines where removed from the first examples. The shell was parsing everything as arguments and removing all extra new lines and spaces.
How to fix this? It is very easy... Just use double quotes so that a single parameter is passed to the echo command:
By putting quotes around the text, it causes the shell to pass the entire text within the quotes as a single argument to echo which preserves the spaces.
This same technique works when echoing a variable that contains the output of a command or the contents of a file:
Hopefully this post helped you understand how commands are parsed by the shell and why you can sometimes see echo commands unexpectedly removing spaces and newlines.
If you've ever tried to "grep" through a Korn shell history file (.sh_history) you've noticed that it doesn't work.. Here is an example showing what happens when you try to do this. I am trying to grep for "rm" in the shell history file. You can "cat" the file and see the content, but when you grep it directly, or pipe cat to grep, nothing but blank lines appear:
The reason why this doesn't work is the .sh_history file is not just a text file, it has some extra non-printable characters in it. You can see some evidence of this in the weird characters shown on the first line of the file when it was displayed with cat. Also, if you do a "cat -v .sh_history" you can see this unprintable characters that are getting in the way of grep:
Also, if you run "file .sh_history" it will report as a "Ultrix-11 Stand-alone or boot executable". So the .sh_history file clearly has extra non-printable characters in it that are messing up our attempt to grep the file.
The solution to this is to use the "strings" utility. strings will look through a binary file and display sections of printable characters out of it. So we can run "strings .sh_history | grep rm" if we want to search through the .sh_history file for "rm"
However, there is one small problem with this... By default "strings" will only print sequences of 4 or more printable characters. So if you ran the command "ls" and then ran "strings .sh_history" it wouldn't show the "ls" line because it is less than 4 characters. The solution to this is to run "strings -n 1 .sh_history | grep rm" which instructs "strings" to display all printable characters and thus every line in your .sh_history file will be searched by grep, regardless of length. In our example it doesn't make a difference since we don't have any commands less than 4 characters, but I just wanted to mention it in case anyone ever wonders why really short commands didn't show up with just the regular "strings .sh_history" command.
|Modified on by brian_s|
|Modified on by brian_s|
This posting is all about return codes, the test command, and the if statement.
Here is a video covering this material as well:
Every command you run in a Linux/UNIX environment has a "return code" when the command completes. A zero (0) return code generally means the command was successful, and non-zero return code generally means failure. You can check the special $? variable to see the return code of the last command that was run.
Here are some examples of commands that return a zero return code (aka success):
$ ls testfile #Listing a file that exists
Here are some examples of commands that have a non-zero return code (aka failure)
$ ls aoeu #Listing a file that doesn't exist
The exclamation point, "!", reverses the return code. If you run "! false", the return code is "0". If you run "! true" the return code is 1. This is useful when you want to test for a situation that is not true.
The test Command:
You don't see the command "test" in scripts very often, but you frequently see statements such as: if [ "$var" == "value" ]; then ... The bracket "[" is just a shortcut for referring to the test command. The if [ "$var" == "value" ]; then ... is equivalent to if test "$var" == "value"; then ... So if you are trying to remember what options you have for comparisons, just run "man test" to list them.
The test command plays by the same rules as other commands, it evaluates the statement and exits with a return code of either zero for success (the statement in question was true), or non-zero for failure (the statement in question was false). Here are some examples, using both the "test" syntax and the bracket "[" syntax:
$ test 4 -lt 6 #Using "test" syntax, is 4 less than 6?
The "if" statement
There is a misconception about the "if" statement in shell scripting. Many people incorrectly think that you always have to do a bracket ("[") comparison when using the if statement (i.e. if [ 5 -lt 6]; then .... ) In fact, the if statement is much more flexible than that. The if statement runs whatever command is specified, and if the result is zero, the if statement code block is run; if the return code is non-zero the if statement code block is skipped.
A lot of scripts use something like this:
A simpler way to do it is just to put the grep command right in the if statement line. The following is simpler and easier to read, and does the exact same thing:
The "ed" editor has for the most part been forgotten as a piece of UNIX history. But it can still be very handy when scripting file edits, especially when the script must work across multiple UNIX variants.
ed is a "line editor". What this basically means is it is designed to work on a teletype (aka keyboard with a printer). This makes it challenging to use as an interactive editor, but it is perfect for scripting. ed is part of the Single UNIX Specification so it should exist on any system claiming to be UNIX, so it is a good utility to rely on in your scripts if they are cross platform.
Many systems such as Linux support the "sed" command with the "-i" flag that specifies the file should be edited in place. Unfortunately systems such as AIX don't support an inline edit mode with sed so you either have to create temporary files (ugly in my opinion) or use something different such as ed.
All ed commands specify a range of lines (or a "," for the entire file), and then a command. For example, within ed you can display lines 1 through 5 with the command "1,5l", or you can display the entire file with ",l".
Here are some example command lines to edit files that can be used from scripts.
First off, we will edit the "sshd_config" file and replace the line "#Port 22" with "Port 222". To do this we use printf to pipe the commands we want to run to ed. In between each ed command we put a "\n" to put a carriage return / new line.
$ cat sshd_config | grep "Port "
So in this example we are telling ed to first run ",s/^#Port 22$/Port222". The first comma means apply to all lines, the "s" means search, the "^#Port 22$" means to look for a line that begins (^) and ends ($) with "Port 22". The "/" means replace, and the "Port 222" is what we want it to be replaced with. The "w" means write the file. The "q" means quit.
Here is an example of adding a line to a file. For example, lets say we want to add the comment line "Updated port to 222 on 8/19/12" to the top of the sshd_config file:
The "1" means to go to line 1, the "i" means insert mode, the "#Updated port to 222 on 8/19/12" is the text we are adding, the "." on the line by itself means to go back to command mode, the "w" is write, and the "q" is quit.
You can also easily delete lines. Lets suppose for example we decide to delete the "Port 222" line from the file:
$ cat sshd_config | grep ^Port
As you can see ed is a powerful utility to edit files from scripts quickly and easily.
Devices in AIX have a child/parent relationship. It can be very helpful to understand this relationship when running commands like rmdev recursively. It can also help to understand the device tree when you run cfgmgr; if you know the device parent you can specify "cfgmgr -l device" which will run faster than a full cfgmgr because it will only scan the parent device. You can find a device parent by running "lsdev -l device -F parent", and you can find child devices of a given device with the command "lsdev -p device"
Below is a script that will produce a full easy to read dependency tree for every device on your AIX system.
If run without the "-d" parameter it doesn't display the details (shows device name only)
Here is the script: