Command substitution is an operation where the shell executes a command (or set of commands), then replaces the command with it's output. In this article we will discuss the basics of how command substitution works, the different syntax styles, their nuances, and of course examine some real world examples.

How Command Substitution Works

Command substitution, in it's most simple form, replaces a command with it's output. Bash does this by running the command in a subshell and replacing the command with it's standard output (STDOUT), and removes any trailing newlines.

Let's take a look at some examples.

Here we will send the output of the date command into an echo string.

[mcherisi@putor ~]$ date +%A
Monday
[mcherisi@putor ~]$ echo "Today is $(date +%A)"
Today is Monday

Command substitution comes in handy in a lot of ways. Here we use it to loop through a directory and create a checksum of each file.

[mcherisi@putor pages]$ for i in $(ls); do md5sum $i; done
695d771e2ecda2dd7ea90100c9027147  dnf-history.html
6d44407d1b3e184f8ae44c73c66c12cc  introduction-to-dev-null.html
1b16dedeeef1a2619d7ecfdd93d6c323  linux-logname-command.html
54b596074f057e064d62b41c518c6ab7  using-trap-to-exit-bash-scripts-cleanly.html

A common use of command substitution is to store the output of a command into a variable. Here we will use grep piped to the wc command to count the number of users who have Bash as their default shell. We will store the output of the command in a variable called BASHUSERS for future use.

[mcherisi@putor ~]$ BASHUSERS=`grep bash /etc/passwd | wc -l`

[mcherisi@putor ~]$ echo "There are $BASHUSERS users with Bash as their default shell."
There are 5 users with Bash as their default shell.

Syntax for Command Substitution

There are two syntax styles you can use for command substitution. The old-style uses backticks (also called backquotes) to wrap the command being substituted. 

`command`

NOTE: A backtick is the key to the left of the number 1 on the keyboard, not to be confused with a single quote.

The new style uses an easier to read, more familiar style. It begins with a dollar sign and wraps the rest of the command in parenthesis.

$(command)

Although these styles are mostly interchangeable, the $(command) style is preferred syntax for the following reasons.

  • $(command) style is POSIX compliant form and therefore likely to work on more shells.
  • backslashes and single quotes are handled in a more familiar manner when using $(command) versus using `command` (more on that below).
  • Readability - The $(...) structure is easy to read and identify in code.
  • Nesting is more convenient in the $(...) style (see the nesting section below)

Backquote Style Command Substitution and Backslashes

The quote below is how the GNU Bash Manual explains the backslash rules for backtick style command substitution.

When the old-style backquote form of substitution is used, backslash retains its literal meaning except when followed by ‘$’, ‘`’, or ‘\’. The first backquote not preceded by a backslash terminates the command substitution. 

- GNU Bash Manual

To me this is unnecessarily vague. In order to make it easier to understand, let's write it like so:

When using the old-style backquote form of substitution, a backslash is a literal backslash; unless followed by a dollar sign, a backquote, or another backslash, in which case it is removed.

Let's do a little experiment to demonstrate how these rules work.

First, let's examine how backslashes normally work. Backslashes are escape characters. They tell bash to treat the character following the backslash literally. Usually, an asterisks would be a wildcard, but since we escaped it with the backslash echo prints the literal character.

[mcherisi@putor ~]$ echo \*
*

Now, for this experiment, let's add an additional backslash to see what happens.

[mcherisi@putor ~]$ echo \\*
\*

As you can see above, the shell escaped the second backslash and the command returned a literal backslash followed by the asterisks.

Now, let's run the same command, this time inside of backticks and redirect the output to a file (otherwise we would pass the output as a command).

[mcherisi@putor ~]$ `echo \\* > output`
[mcherisi@putor ~]$ cat output
*

This time, the command inside the backticks is evaluated according to the backtick quoting rules. The rules remove any backslash that is followed by a "$", "", or a "\". Since the first backslash is followed by another backslash, it is removed. As asterisks follows the second backslash, so it remains. This breaks it down to echo \*. Now the output of echo \* is a single asterisks (just as in the first step in this experiment).

New Style (POSIX-Compliant) Command Substitution

When using the new style $(...) all characters between the parenthesis are used to make up the command. Let's go back to our example.

Regular backslash escaping:

[mcherisi@putor ~]$ echo \\*
\*

Now with the new style command substitution:

[mcherisi@putor ~]$ $(echo \\* > output)
[mcherisi@putor ~]$ cat output
\*

Now, what is inside the parenthesis is evaluated as \\*. The main difference is the first backslash (inside the parenthesis) is NOT treated as an escape. So we send \\* to the echo command. It then undergoes the normal shell escaping rules (because it is outside of the parenthesis), which escapes the second backslash with the first and the result is \*.

Nested Command Substitution

Nesting is when you have a command inside of a similar command. For example, a nested if statement is when you have an if statement inside of another if statement. Here we will examine how to use nested command substitutions in both styles and why one is commonly preferred over the other.

Nested Backquote Command Substitution

Let's look at nested backquote style command substitution first. These two lines from the Bash Manual explain how to use nesting with the backquoted form.

The first backquote not preceded by a backslash terminates the command substitution....
...To nest when using the backquoted form, escape the inner backquotes with backslashes.

- GNU Bash Manual

Here is an example of nested backquote style command substitution:

[mcherisi@putor ~]$ me=`cat /etc/passwd | grep \`whoami\``

[mcherisi@putor ~]$ echo $me
mcherisi:x:1005:1005::/home/mcherisi:/bin/bash

In a simple example like this, it doesn't look too bad. But you can imagine how that would get harder to read as the levels of nesting grow. The backtick is also hard on the human eye, as it looks very much like a single quote. This can cause mistakes when looking at large amounts of code.

Nested New Style $() Command Substitution

Using escape characters isn't the only thing that is easier with the new style. Nesting is also much more convenient and easier to read. Since the opening and closing characters are different, it provides a simple way to nest without having to escape any characters.

Here is an example of nested new $(...) style command substitution:

[mcherisi@putor ~]$ me=$(cat /etc/passwd | grep $(whoami) )

[mcherisi@putor ~]$ echo $me
mcherisi:x:1005:1005::/home/mcherisi:/bin/bash

This is much cleaner and easier on the eyes!

Conclusion

In this article we covered both styles of command substitution. We also discussed the differences of how each style treats backslashes. We then moved on to nesting and why the new style is preferred over the old backtick style. I hope you enjoyed this article and are now comfortable using command substitution in your Bash scripts. Once you are familiar with it, you will continue to find new and interesting uses.

As always, we welcome any questions or comments. Ciao!

Resources and Links