Skip to content

Expr Shell Variable Assignment

5.9. Math in Shell Scripts¶

Shell script variables are by default treated as strings, not numbers, which adds some complexity to doing math in shell script. To keep with script programming paradigm and allow for better math support, languages such Perl or Python would be better suited when math is desired. However, it is possible to do math with shell script. In fact, over the years, multiple facilities have been added to Unix to support working with numbers.

Note

As we will see, some of the commands used to facilitate math are a little picky about things like spaces around operators.

5.9.1. declare

You may recall, that when the text book introduced the declare statement, it said that it is not always needed. So what do you get by declaring a variable to be an integer? The following example illustrates that a declared integer is not treated as a string.

When you do not need the declare statement is when you will use a program or built-in command to evaluate a math statement.

$ n=6/3 $ echo $n 6/3 $ declare -i n $ n=6/3 $ echo $n 2

5.9.2. expr

An old Unix program that can evaluate math is expr. expr became popular in the days of the Bourne Shell, which did not support math. With Bash and Korn shell, it is generally not needed. Since it is a command, command substitution is needed. We are still treating the variable as a string. As you can see, it is picky about spaces.

$ z=5 $ z=`expr $z+1` ---- Need spaces around + sign. $ echo $z 5+1 $ z=`expr $z + 1` $ echo $z 6

5.9.3. let

A Bash and Korn shell built-in command for math is let. As you can see, it is also a little picky about spaces, but it wants the opposite of what expr wanted. let also relaxes the normal rule of needing a $ in front of variables to be read.

$ let z=5 $ echo $z 5 $ let z=$z+1 $ echo $z 6 $ let z=$z + 1 # --- Spaces around + sign are bad with let -bash: let: +: syntax error: operand expected (error token is "+") $let z=z+1 # --- look Mom, no $ to read a variable. $echo $z 7

5.9.4. BASH Arithmetic¶

With the BASH shell, whole arithmetic expressions may be placed inside double parenthesis. This form is more forgiving about spaces.

Addition, subtration
Increment, decrement
Multiplication, division, remainder
Exponentiation

Numerical boolean expressions in Control Constructs are also expressed using double parenthesis.

Less than or equal, greater than or equal, less than, greater than
Equal, not equal
Logical NOT
Logical AND
Logical OR
$ ((e=5)) $ echo $e 5 $ (( e = e + 3 )) $ echo $e 8 $ (( e=e+4 )) # -- spaces or no spaces, it doesn't matter $ echo $e 12
if((x>y));thenecho"x > y"fi

5.9.5. bc

What if you want to do math with floating point numbers or you have some fairly complicated math to do? Neither form of let, supports floating point numbers. The bc command is needed. But you have to treat the variables as strings.

Here is what happens when we try to do floating point math with the shell:

An arbitrary precision calculator language. bc may either be run interactively, or as a shell script command. In interactive mode, type cntrl-d (EOF) to exit.

SYNOPSIS

Here are some examples:

Remember to type cntrl-d (EOF) to exit from interactive mode.

$let r=3.5 -bash: let: r=3.5: syntax error in expression (error token is ".5") $(( r = 3.5 )) -bash: ((: r = 3.5 : syntax error in expression (error token is ".5 ")
$ bc bc 1.06 Copyright 1991-1994, 1997, 1998, 2000 Free Software Foundation, Inc. This is free software with ABSOLUTELY NO WARRANTY. For details type `warranty'. 3 + 2 5 obase=2 12 1100 <cntrl-d>
$r=3.5 $s=`echo "$r + 2.2" | bc` $echo $s 5.7 $ z = `echo $z + 1 | bc` -bash: z: command not found # -- spaces around = sign are bad (shell thing, not bc) $ z=`echo "$z + 1" | bc` $ echo $z 8 $ z=`echo "$z+1" | bc` -- spaces don't matter with bc $ echo $z 9

5.9.6. Numeric Boolean expressions¶

If BASH double parenthesis are not used, then the test command must be used to compare integer variables. See test.

INTEGER1 -eq INTEGER2
INTEGER1 is equal to INTEGER2
INTEGER1 -ge INTEGER2
INTEGER1 is greater than or equal to INTEGER2
INTEGER1 -gt INTEGER2
INTEGER1 is greater than INTEGER2
INTEGER1 -le INTEGER2
INTEGER1 is less than or equal to INTEGER2
INTEGER1 -lt INTEGER2
INTEGER1 is less than INTEGER2
INTEGER1 -ne INTEGER2
INTEGER1 is not equal to INTEGER2

That is to say, the following two statements are identical:

And for logical expressions using floating point math, the bc command returns 1 for logical true expressions and 0 for false expressions, thus testing for result of 1 achieves a logical expression:

if (( x < y )); then statements fi if [ $x -lt $y ]; then statements fi
if [ $( echo "$t < 3.4" | bc ) -eq 1 ]; then statements fi

Bash variables and command substitution

An essential feature of programming is the ability to use a name or a label to refer to some other quantity: such as a value, or a command. This is commonly referred to as variables.

Variables can be used, at the very least, to make code more readable for humans:

However, variables really come into use in more advanced programming, when we're in a situation in which the actual values aren't known before executing a program. A variable acts as a placeholder that gets resolved upon actual execution time.

For example, imagine that contains a list of website addresses. The following routine reads each line (via , which isn't best practice…but will do for now) into a loop, which then downloads each URL:

Variables

Basic variable usage and syntax

Setting a variable

The following command assigns to the variable named , and to

Unlike most modern languages, Bash is pretty picky about the syntax for setting variables. In particular, no whitespace is allowed between the variable name, the equals sign, and the value.

All of these examples would cause Bash to throw an error:

Referencing the value of a variable

Whenever Bash encounters a dollar-sign, immediately followed by a word, within a command or in a double-quoted string, it will attempt to replace that token with the value of the named variable. This is sometimes referred to as expanding the variable, or parameter substitution:

Failure to dereference

When a dollar-sign doesn't precede a variable name, or a variable reference is within single-quotes, Bash will interpret the string literally:

Concatenating strings

Variables can be very useful for text-patterns that will be repeatedly used:

If your variable name butts up against a literal alphanumeric character, you can use this more verbose form, involving curly braces, to reference a variable's value:

Valid variable names

Variable names can contain a sequence of alphanumeric characters and underscores. For variables created by you, the user, they should start with either an alphabetical letter or an underscore (i.e. not a number):

Valid variable names:

    When we write functions and shell scripts, in which arguments are passed in to be processed, the arguments will be passed int numerically-named variables, e.g. , ,

    For example:

    Inside , commands will use to refer to , to , and for

    The variable reference, , will expand to the current script's name, e.g.


    Command substitution

    The standard output of a command can be encapsulated, much like a value can be stored in a value, and then expanded by the shell.

    This is known as command substitution. From the Bash documentation:

    Command substitution allows the output of a command to replace the command itself. Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.

    As an example, consider the seq command, which will print a sequence of numbers starting from the first argument to the second argument:

    With command substitution, we can encapsulate the result of into a variable by enclosing the command with and , and pass it as an argument to another command:

    As a GIF:

    Variables and command expansion

    When a command is replaced by its standard output, that output, presumably just text, can be assigned to a variable like any other value:

    The loss of newlines in command substitution

    Earlier, I quoted from the Bash documentation on command expansion. Here's an emphasized version of the excerpt:

    Command substitution allows the output of a command to replace the command itself. Bash performs the expansion by executing command and replacing the command substitution with the standard output of the command, with any trailing newlines deleted. Embedded newlines are not deleted, but they may be removed during word splitting.

    What does that mean? Consider being called normally, and then, via command substitution, and note the change in formatting:

    Why do the newlines get removed during the command expansion? It's something we'll encounter later (and there's a section on it at the end of this tutorial) and deals with how Bash interprets space and newline characters during expansion. Anyway, it's worth noting the behavior for now, as it may be new to you if you're coming from another programming language.

    Arithmetic expansion

    To do basic calculations, you can enclose an expression inside :

    Check the Bash documentation for the full set of arithmetic operators. Math at the command-line can be a bit clunky so we won't be focusing too much on it.

    The bc utility

    An aside: if you want to do more advanced math from the command line, use bc, which reads in from stdout and evaluates the expression:

    Word-splitting in the wild

    This section covers more technical details of how Bash handles space characters when it does an exapansion. It's not necessary to memorize for the specific assignments in this class. However, as many of you are wont to copy and paste code directly from things you've seen on the Internet, it might be worth knowing all the different ways you could accidentally harm yourself, due to the way Bash handles spaces and newline characters.

    Here's the Bash documentation for the concept known as "word-splitting"

    The internal field separator

    The global variable is what Bash uses to split a string of expanded into separate words…think of it as how Excel knows to split a CSV (comma-separated-values) text file into a spreadsheet: it assumes the commas separate the columns.

    Let's pretend that has been set to something arbitrary, like a capital . When Bash expands a variable that happens to contain a , the value of that variable will be split into separate words (and the literal will disappear):

    By default, the variable is set to three characters: newline, space, and the tab. If you , you won't see anything because those characters…well, how do you see a space character if there aren't any visible characters?

    The upshot is that you may see code snippets online in which the variable is changed to something like (which stands for the newline character).

    Imagine a textfile that contains a bunch of lines of text that, for example, may refer to filenames:

    When Bash reads each line of the file, the default value of , which includes a space character, will cause Bash to treat the file named as two files, and , because the space character is used to split words.

    With set to just the newline character, is treated as a single filename.

    This concept will make sense when it comes to reading text files and operating on each line. I don't expect you to fully understand this, but only to be aware of it, just in case you are haphazardly copy-pasting code from the Internet.

    The dangers of unquoted variables

    In an ideal world, everyone would keep their string values short and without space/newline, or any other special characters. In that ideal world, the following unquoted variable reference would work just fine:

    But when people start adding special characters to filenames, such as spaces, expanding variables, without the use of double quotes, can be dangerous.

    In the following example, the programmer intends the file named to be deleted:

    Unexpected word-splitting

    However, when referenced without double-quotes, Bash sees as containing two separate values, and . The subsequent command will attempt to delete those two files, and not :

    Unexpected special characters in filenames

    Ah, no harm done, you say, because those files didn't exist in the first place. OK, but what happens when someone puts a star (i.e. asterisk) into a filename? You're aware of what happens when you do and – the star acts as a wildcard, grabbing every file.

    So you'll see the previous errors, since and don't exist. But in between those attempted deletions, will run on …so say bye-bye to every file in that directory.

    Here's the animated GIF version:

    Notice how affects only the file that is named, .

    So the main takeaway here is: double-quote your variable references whenever possible.

    To reiterate

    Expanding a variable can lead to unexpected and sometimes catastrophic results if the variable contains special characters:

    Expanding a variable within double-quotes can prevent such problems:

    Who would do such a thing?

    You might think, Who the hell puts star characters in their filenames? Well, besides people who really enjoy star-shaped symbols, malicious hackers and pranksters. And variables usually aren't just manually assigned by the result of human typing. As you've read above, sometimes the result of commands are stored in a variable. And if such commands are processing raw data, it's not unimaginable that the raw data, quite innocently, contains special characters that are destructive to certain Bash programs.

    For the purposes of the CompCiv course, the assignments will try to stay far from untrusted sources of data. But keep in mind the dangers of just pasting in seemingly safe-looking code. Bash's syntax and behavior in handling strings is hard to fully comprehend, which is why developers use other languages for more complex applications.

    You can read more about quoting variables. There's a lot of minutiae, but the main takeaway, besides general safety, is to have a general understanding how Bash, and any other programming environment, uses certain conventions and syntax rules to deal with the myriad ways that users want to pass around values in their programs.