One common problem I have personally made and seen others make while shell scripting is trying to set a variable to be the contents of a file or the output of a multiline command, and then trying to echo the variable to process it further with grep, awk, a while loop, etc. Depending on how you do this you might get some unexpected results because you might have missing newlines and spaces from your output.
In this example, we have a file named "testfile" that contains 5 lines. I then set the testvar variable to contain the contents of the line by running "testvar=`cat testfile`". However, when I attempt to "echo $testvar" all the lines of the file are shown on one line! If I try to grep for "test line 2" it still see's everything on one line.
The same problem happens when trying to set testvar to be the output of "ls -al" and then echoing the variable:
So what is going on here? It all has to do with how command line parameters are parsed by the shell and provided to the echo command.
Here is an example that illustrates what is happening:
Notice how the echo command removed all the extra spaces out of what I had typed? It is doing this because the shell is parsing each word as an argument and providing the arguments to the echo command:
The shell is parsing arguments and providing them to the echo command. As part of parsing the arguments, all of the extra spaces are being removed by the shell before they are passed to the echo command. This is the same thing that was happening when our newlines where removed from the first examples. The shell was parsing everything as arguments and removing all extra new lines and spaces.
How to fix this? It is very easy... Just use double quotes so that a single parameter is passed to the echo command:
By putting quotes around the text, it causes the shell to pass the entire text within the quotes as a single argument to echo which preserves the spaces.
This same technique works when echoing a variable that contains the output of a command or the contents of a file:
Hopefully this post helped you understand how commands are parsed by the shell and why you can sometimes see echo commands unexpectedly removing spaces and newlines.