Command-line Environment
Job Control
In some cases you will need to interrupt a job while it is executing, for instance if a command is taking too long to complete (such as a find
with a very large directory structure to search through). Most of the time, you can do Ctrl-C
and the command will stop. But how does this actually work and why does it sometimes fail to stop the process?
Killing a process
Your shell is using a UNIX communication mechanism called a signal to communicate information to the process. When a process receives a signal it stops its execution, deals with the signal and potentially changes the flow of execution based on the information that the signal delivered. For this reason, signals are software interrupts.
In our case, when typing Ctrl-C
this prompts the shell to deliver a SIGINT
signal to the process.
Here's a minimal example of a Python program that captures SIGINT
and ignores it, no longer stopping. To kill this program we can now use the SIGQUIT
signal instead, by typing Ctrl-\
.
Here's what happens if we send SIGINT
twice to this program, followed by SIGQUIT
. Note that ^
is how Ctrl
is displayed when typed in the terminal.
Pausing and backgrounding processes
Signals can do other things beyond killing a process. For instance, SIGSTOP
pauses a process. In the terminal, typing Ctrl-Z
will prompt the shell to send a SIGTSTP
signal, short for Terminal Stop (i.e. the terminal's version of SIGSTOP
).
One more thing to know is that the &
suffix in a command will run the command in the background, giving you the prompt back, although it will still use the shell's STDOUT which can be annoying (use shell redirections in that case).
Below is a sample session to showcase some of these concepts.
A special signal is SIGKILL
since it cannot be captured by the process and it will always terminate it immediately. However, it can have bad side effects such as leaving orphaned children processes.
Terminal Multiplexers
When using the command line interface you will often want to run more than one thing at once. For instance, you might want to run your editor and your program side by side. Although this can be achieved by opening new terminal windows, using a terminal multiplexer is a more versatile solution.
tmux
expects you to know its keybindings, and they all have the form <C-b> x
where that means (1) press Ctrl+b
, (2) release Ctrl+b
, and then (3) press x
. tmux
has the following hierarchy of objects:
Sessions - a session is an independent workspace with one or more windows
tmux
starts a new session.tmux new -s NAME
starts it with that name.tmux ls
lists the current sessionsWithin
tmux
typing<C-b> d
detaches the current sessiontmux a
attaches the last session. You can use-t
flag to specify which
Windows - Equivalent to tabs in editors or browsers, they are visually separate parts of the same session
<C-b> c
Creates a new window. To close it you can just terminate the shells doing<C-d>
<C-b> N
Go to the N th window. Note they are numbered<C-b> p
Goes to the previous window<C-b> n
Goes to the next window<C-b> ,
Rename the current window<C-b> w
List current windows
Panes - Like vim splits, panes let you have multiple shells in the same visual display.
<C-b> "
Split the current pane horizontally<C-b> %
Split the current pane vertically<C-b> <direction>
Move to the pane in the specified direction. Direction here means arrow keys.<C-b> z
Toggle zoom for the current pane<C-b> [
Start scrollback. You can then press<space>
to start a selection and<enter>
to copy that selection.<C-b> <space>
Cycle through pane arrangements.
Aliases
It can become tiresome typing long commands that involve many flags or verbose options. For this reason, most shells support aliasing. A shell alias is a short form for another command that your shell will replace automatically for you. For instance, an alias in bash has the following structure:
Aliases have many convenient features:
Note that aliases do not persist shell sessions by default. To make an alias persistent you need to include it in shell startup files, like .bashrc
or .zshrc
, which we are going to introduce in the next section.
Dotfiles
Many programs are configured using plain-text files known as dotfiles (because the file names begin with a .
, e.g. ~/.vimrc
, so that they are hidden in the directory listing ls
by default).
For bash
, editing your .bashrc
or .bash_profile
will work in most systems. Here you can include commands that you want to run on startup, like the alias we just described or modifications to your PATH
environment variable. In fact, many programs will ask you to include a line like export PATH="$PATH:/path/to/program/bin"
in your shell configuration file so their binaries can be found.
Some other examples of tools that can be configured through dotfiles are:
bash
-~/.bashrc
,~/.bash_profile
git
-~/.gitconfig
vim
-~/.vimrc
and the~/.vim
folderssh
-~/.ssh/config
tmux
-~/.tmux.conf
How should you organize your dotfiles? They should be in their own folder, under version control, and symlinked into place using a script. This has the benefits of:
Easy installation: if you log in to a new machine, applying your
customizations will only take a minute.
Portability: your tools will work the same way everywhere.
Synchronization: you can update your dotfiles anywhere and keep them all
in sync.
Change tracking: you're probably going to be maintaining your dotfiles
for your entire programming career, and version history is nice to have for
long-lived projects.
Portability
A common pain with dotfiles is that the configurations might not work when working with several machines, e.g. if they have different operating systems or shells. Sometimes you also want some configuration to be applied only in a given machine.
There are some tricks for making this easier. If the configuration file supports it, use the equivalent of if-statements to apply machine specific customizations. For example, your shell could have something like:
If the configuration file supports it, make use of includes. For example, a ~/.gitconfig
can have a setting:
And then on each machine, ~/.gitconfig_local
can contain machine-specific settings. You could even track these in a separate repository for machine-specific settings.
This idea is also useful if you want different programs to share some configurations. For instance, if you want both bash
and zsh
to share the same set of aliases you can write them under .aliases
and have the following block in both:
Remote Machines
It has become more and more common for programmers to use remote servers in their everyday work. If you need to use remote servers in order to deploy backend software or you need a server with higher computational capabilities, you will end up using a Secure Shell (SSH). As with most tools covered, SSH is highly configurable so it is worth learning about it.
To ssh
into a server you execute a command as follows
Here we are trying to ssh as user foo
in server bar.mit.edu
. The server can be specified with a URL (like bar.mit.edu
) or an IP (something like foobar@192.168.1.42
). Later we will see that if we modify ssh config file you can access just using something like ssh bar
.
Executing commands
An often overlooked feature of ssh
is the ability to run commands directly. ssh foobar@server ls
will execute ls
in the home folder of foobar. It works with pipes, so ssh foobar@server ls | grep PATTERN
will grep locally the remote output of ls
and ls | ssh foobar@server grep PATTERN
will grep remotely the local output of ls
.
SSH Keys
Key-based authentication exploits public-key cryptography to prove to the server that the client owns the secret private key without revealing the key. This way you do not need to reenter your password every time. Nevertheless, the private key (often ~/.ssh/id_rsa
and more recently ~/.ssh/id_ed25519
) is effectively your password, so treat it like so.
Key generation
Key based authentication
ssh
will look into .ssh/authorized_keys
to determine which clients it should let in. To copy a public key over you can use:
A simpler solution can be achieved with ssh-copy-id
where available:
Copying files over SSH
There are many ways to copy files over ssh:
Port Forwarding
In many scenarios you will run into software that listens to specific ports in the machine. When this happens in your local machine you can type localhost:PORT
or 127.0.0.1:PORT
, but what do you do with a remote server that does not have its ports directly available through the network/internet?.
Local Port Forwarding
Remote Port Forwarding
The most common scenario is local port forwarding, where a service in the remote machine listens in a port and you want to link a port in your local machine to forward to the remote port. For example, if we execute jupyter notebook
in the remote server that listens to the port 8888
. Thus, to forward that to the local port 9999
, we would do ssh -L 9999:localhost:8888 foobar@remote_server
and then navigate to locahost:9999
in our local machine.
SSH Configuration
We have covered many many arguments that we can pass. A tempting alternative is to create shell aliases that look like
However, there is a better alternative using ~/.ssh/config
.
An additional advantage of using the ~/.ssh/config
file over aliases is that other programs like scp
, rsync
, mosh
, &c are able to read it as well and convert the settings into the corresponding flags.
Note that the ~/.ssh/config
file can be considered a dotfile, and in general it is fine for it to be included with the rest of your dotfiles. However, if you make it public, think about the information that you are potentially providing strangers on the internet: addresses of your servers, users, open ports, &c. This may facilitate some types of attacks so be thoughtful about sharing your SSH configuration.
Server side configuration is usually specified in /etc/ssh/sshd_config
. Here you can make changes like disabling password authentication, changing ssh ports, enabling X11 forwarding, &c. You can specify config settings on a per user basis.
Miscellaneous
Shells & Frameworks
During shell tool and scripting we covered the bash
shell because it is by far the most ubiquitous shell and most systems have it as the default option. Nevertheless, it is not the only option.
For example, the zsh
shell is a superset of bash
and provides many convenient features out of the box such as:
Smarter globbing,
**
Inline globbing/wildcard expansion
Spelling correction
Better tab completion/selection
Path expansion (
cd /u/lo/b
will expand as/usr/local/bin
)
Right prompt
Command syntax highlighting
History substring search
manpage based flag completions
Smarter autocompletion
Prompt themes
One thing to note when using these frameworks is that they may slow down your shell, especially if the code they run is not properly optimized or it is too much code. You can always profile it and disable the features that you do not use often or value over speed.
Terminal Emulators
Since you might be spending hundreds to thousands of hours in your terminal it pays off to look into its settings. Some of the aspects that you may want to modify in your terminal include:
Font choice
Color Scheme
Keyboard shortcuts
Tab/Pane support
Scrollback configuration
Exercises
Job control
However, this strategy will fail if we start in a different bash session, since
wait
only works for child processes. One feature we did not discuss in the notes is that thekill
command's exit status will be zero on success and nonzero otherwise.kill -0
does not send a signal but will give a nonzero exit status if the process does not exist. Write a bash function calledpidwait
that takes a pid and waits until the given process completes. You should usesleep
to avoid wasting CPU unnecessarily.
Terminal multiplexer
Aliases
Create an alias
dc
that resolves tocd
for when you type it wrongly.Run
history | awk '{$1="";print substr($0,2)}' | sort | uniq -c | sort -n | tail -n 10
to get your top 10 most used commands and consider writing shorter aliases for them. Note: this works for Bash; if you're using ZSH, usehistory 1
instead of justhistory
.
Dotfiles
Remote Machines
Edit
.ssh/config
to have an entry as follows
Use
ssh-copy-id vm
to copy your ssh key to the server.Start a webserver in your VM by executing
python -m http.server 8888
. Access the VM webserver by navigating tohttp://localhost:9999
in your machine.Edit your SSH server config by doing
sudo vim /etc/ssh/sshd_config
and disable password authentication by editing the value ofPasswordAuthentication
. Disable root login by editing the value ofPermitRootLogin
. Restart thessh
service withsudo service sshd restart
. Try sshing in again.(Challenge) Look into what the
-N
and-f
flags do inssh
and figure out what a command to achieve background port forwarding.
Last updated
Was this helpful?