Shell Tools and Scripting
Shell Scripting
Bash Basics
To assign variables in bash, use the syntax foo=bar
and access the value of the variable with $foo
. Note that foo = bar
will not work since it is interpreted as calling the foo
program with arguments =
and bar
. In general, in shell scripts the space character will perform argument splitting. This behavior can be confusing to use at first, so always check for that.
Strings in bash can be defined with '
and "
delimiters, but they are not equivalent. Strings delimited with '
are literal strings and will not substitute variable values whereas "
delimited strings will.
As with most programming languages, bash supports control flow techniques including if
, case
, while
and for
. Similarly, bash
has functions that take arguments and can operate with them. Here is an example of a function that creates a directory and cd
s into it.
Here $1
is the first argument to the script/function.
Bash Special Variables
$0
- Name of the script$1
to$9
- Arguments to the script.$1
is the first argument and so on.$@
- All the arguments$#
- Number of arguments$?
- Return code of the previous command$$
- Process identification number (PID) for the current script!!
- Entire last command, including arguments. A common pattern is to execute a command only for it to fail due to missing permissions; you can quickly re-execute the command with sudo by doingsudo !!
$_
- Last argument from the last command. If you are in an interactive shell, you can also quickly get this value by typingEsc
followed by.
Return Codes
The return code or exit status is the way scripts/commands have to communicate how execution went. A value of 0 usually means everything went OK; anything different from 0 means an error occurred.
Command and Process Substitution
Another common pattern is wanting to get the output of a command as a variable. This can be done with command substitution. Whenever you place $( CMD )
it will execute CMD
, get the output of the command and substitute it in place. For example, if you do for file in $(ls)
, the shell will first call ls
and then iterate over those values. A lesser known similar feature is process substitution, <( CMD )
will execute CMD
and place the output in a temporary file and substitute the <()
with that file’s name. This is useful when commands expect values to be passed by file instead of by STDIN. For example, diff <(ls foo) <(ls bar)
will show differences between files in dirs foo
and bar
.
Globbing
Wildcards - Whenever you want to perform some sort of wildcard matching, you can use
?
and*
to match one or any amount of characters respectively. For instance, given filesfoo
,foo1
,foo2
,foo10
andbar
, the commandrm foo?
will deletefoo1
andfoo2
whereasrm foo*
will delete all butbar
.Curly braces
{}
- Whenever you have a common substring in a series of commands, you can use curly braces for bash to expand this automatically. This comes in very handy when moving or converting files.
Shellcheck
Shebangs
Note that scripts need not necessarily be written in bash to be called from the terminal. For instance, here’s a simple Python script that outputs its arguments in reversed order:
Shell Functions vs Scripts
Functions have to be in the same language as the shell, while scripts can be written in any language. This is why including a shebang for scripts is important.
Functions are loaded once when their definition is read. Scripts are loaded every time they are executed. This makes functions slightly faster to load, but whenever you change them you will have to reload their definition.
As with any programming language, functions are a powerful construct to achieve modularity, code reuse, and clarity of shell code. Often shell scripts will include their own function definitions.
Shell Tools
Finding how to use commands
man
, -h
, --help
,:help
,?
, TLDR pages: tldr
Finding files
Beyond listing files, find can also perform actions over files that match your query. This property can be incredibly helpful to simplify what could be fairly monotonous tasks.
Finding code
grep
has many flags that make it a very versatile tool. Some I frequently use are -C
for getting Context around the matching line and -v
for inverting the match, i.e. print all lines that do not match the pattern. For example, grep -C 5
will print 5 lines before and after the match. When it comes to quickly searching through many files, you want to use -R
since it will Recursively go into directories and look for files for the matching string.
Finding shell commands
The history
command will let you access your shell history programmatically. It will print your shell history to the standard output. If we want to search there we can pipe that output to grep
and search for patterns. history | grep find
will print commands that contain the substring “find”.
Lastly, a thing to have in mind is that if you start a command with a leading space it won’t be added to your shell history. This comes in handy when you are typing commands with passwords or other bits of sensitive information. If you make the mistake of not adding the leading space, you can always manually remove the entry by editing your .bash_history
or .zhistory
.
Directory Navigation
Exercises
Last updated
Was this helpful?