As a follow up to my top 10 unix tips article, I've put together a list of odd 'gotchas' (things that people get caught out by) for working on UNIX systems in here too: There's some weird stuff in here, but all are things that I've seen trip up several people in the past. Some are more common (generally the ones nearer the top of the list), others are more obscure. All are worth knowing about!
- find . -name *.txt This command is a bad idea. The '*' needs to be processed by the find command, and not by the shell. Prevent shell expansion of the wildcard by either running 'find . -name \*.txt' or 'find . -name "*.txt"'. If you don't do one of those things, and there is a file matching *.txt in the current directory, then the command will only display files with that specific name, not everything in subdirectores matching the wildcard, or if there are multiple matches in the current directory, the find command will fail. Try it :)
- My terminal's locked up!!! In most cases this is as a result of hitting the XOFF flow control sequence CTL-S, which pauses a terminal unless the forgorund application has overridden it somehow. Press CTL-Q to release it (XON).
- My backround processes disappear when I log out By default, when you terminate a shell, any background processes (started by running a command with & at the end) get sent a SIGHUP signal that causes them to die. If you want them to continue running after you log out, prefix the command with 'nohup' which starts the process in a way that ignores the SIGHUP. Alternatively, use "screen" as described in my 'top 10 unix tips' article.
- X applications don't work after I run 'su -'Access to your X server (display) is controlled via a file called .Xauthority in your home directory. This file contains entries called cookies which control which displays you have access to. If you 'su -' to another account, you lose your environment variables, including your home directory as specified in the HOME variable (which is changed to that of the target user) and the DISPLAY variable is also lost. You can resolve this either (1) using 'su' on it's own, which won't change the variables, assuming the target user has access to your .Xauthority (root usually will, no-one else should), (2) Using xauth as described in my unix tips entry or (3) fudging the .Xauthority file, which I won't explain any further because it's a bad idea :-p
- ssh keys don't workIf your ssh keys aren't working i.e. you can't automatically log into a system even though you've put the correct keys in place, then check the permissions on the files containing the keys. They should not be readable by anyone other than the owner and root. Also check that none of the directories in the path from / down to the directory containing the key file are writable by anyone other than the user. If you're using a distributed file system such as AFS you may not be able to resolve this. Another thing worth checking if you have this problem is if any of the directories down to your home directory is owned by a non-root user. I recently had a system where / was owned by a normal user, so no-one else's keys would work.
- Command hashing. Particularly in bash, it's possible to fool the shell so that you end up executing the wrong version of a command because it caches everything. If you have two commands with the same name in your PATH, run the first, then delete the program, and run it again you won't pick up the second version but will get an error saying the command doesn't exist. Likewise if you add another command with the same name to an earlier location in your PATH it won't get picked up. The shell caches the location of commands on your PATH to avoid looking them up each time - use 'hash' to display the contents of the cache. ksh doesn't usually suffer the same problems because it doesn't seem to cache everything, but they can always be resolved with 'hash -r' which flushes the cache
- Variables can't be set inside while loops If you run a command and pipe it into a while loop, such as 'cat file.txt | while read LINE; do ...; done' then any variables you set within the do/done section won't be available outside the loop later on. This is because the pipe causes the while to execute in a subshell, so any variables set in there won't affect the original shell. I've seen this trip up a few people in the past :)
- I've deleted a file, but haven't recovered the space! There are three possible reasons for this. (1) Another process still has the file opened (UNIX doesn't prevent you deleting in-use files from the directory entries, but the files will still exist until all applications have closed them) (2) The file is full of holes (that would take too long to explain but ask if you're interested!) (3) The file was a hard link to another file, in which case you've just removed a reference to it, not the underlying file. Hard link counts are shown in second column of the 'ls -l' output. The file isn't removed until the link count is zero.
- Help! I can't see my files! Create a directory, change into it, then from another shell delete the directory, recreate it, and put stuff in it. You won't be able to see the new files from the original shell, and commands like /bin/pwd will give errors along the lines of 'cannot determine current directory!'. Things aren't really deleted on UNIX as long as something's using them (e.g. a shell sitting in the directory), so the old directory is still there until the original shell changes out of it. In bash, the rather bizarre looking 'cd $PWD' from the original shell will switch you into the new one. In ksh you have to explicitly 'cd' to another directory then back (or 'cd -')
- My while loop isn't looping/My nohup'd jobs don't keep running! This one's included simply because today I spent more time than is sensible debugging it - and it's not the first time I've hit this problem 'cat file | while read FIELD; do echo Searching for $FIELD; grep $FIELD $TARGET; done'. Problem? If $TARGET isn't defined - most likely because you mistyped the variable name - the missing parameter will mean that grep will be waiting on input from the console, at which point the loop will abort. Solution - if you must have a command that reads from stdin, add '</dev/null' to the end of the command. This will also solve a similar problem which can occur with commands executed with nohup that stop with a SIGTTIN signal.
I hope you've found this educational, and hopefully it'll reduce some confusion for some of you in the future.