Chaining and piping at the command line

At the command line, you will find that you frequently need to run multiple commands to accomplish your end goal. It’s important, therefore, to understand the various options you have to link commands together:

Consecutively

The simplest way is to run multiple commands is to connect them with semi-colons:

command1 ; command2 ; command3

Doing this will run command 1, then command 2, then command 3, consecutively. It will wait for the previous command to complete before proceeding with the next one.

AND-ed

Alternately, you can connect commands with two ampersands between each, which will add a logic component to the operation by AND-ing them together. This runs the commands consecutively, but has the additional effect of will paying attention to how each command exits, and only proceeds to the next on success of the previous. So, for example:

command1 && command2

This will run command 1, and if that is successful, command 2 will run. If command 1 fails, command 2 will not be run.

OR-ed

On the flip side, if you OR them together with two pipes, you flip the logic so that it only proceeds on failure:

command1 || command2

In this case, command 2 will run only if command 1 does not successfully complete. This is useful if you need to log something on failure, for instance.

Grouping

Additionally, you can utilize parentheses to group commands together, just like mathematical operations:

command1 || (command2 && command3)

This will run command 1, and if it fails, command 2 will run, and on its success, command 3.

In Parallel

If the running of commands don’t each depend on the previous one having been run successfully, you can connect them with ampersand to run them in parallel, which will let you complete them all in a shorter amount of time:

command1 & command2 & command3

When you run these commands in parallel, they are all run simultaneously in the background. The system will assign each job in the run an ID and will output a list of which job got what ID:

[1] 10945
[2] 10946
[3] 10947

Knowing these IDs is handy in case you want to stop one by killing it. You can kill a job by its ID number:

kill 19945

or via its index number in the job list:

kill %1

Additionally can check on the progress of any job in the list issuing the foreground command following by its index number:

fg %1

Running this will bring the first job into the foreground, and you’ll see any output it’s generating in your terminal window. To put it back into the background, hit ctrl-z to suspend it and give you the command line again, and then enter :

bg %1

…to resume its running in the background again. That’s the difference between ctrl-c and ctrl-z – ctrl-c will kill a process with a SIGINT, and ctrl-z will merely suspend it with a SIGSTOP, which lets you continue it at a later time.

As each job completes, it outputs a line saying that it’s done:

[1] Done command1

If, during their runtime, if you need to see which job is which, and what their current runstates are, just enter the command

jobs

…and you’ll get a list of what’s running:

[1] Running command1 &
[2]- Running command2 &
[3]+ Running command3 &

Piping

Finally, by gluing commands together with a single pipe, you can pass the output of command to subsequent commands. This is extremely powerful for getting stuff done.

For instance, to find all of the files in a filelist that contain the term foo, you could redirect the output of ls to a text file:

ls > bar.txt

…and then grep on it:

grep foo bar.txt

But you can save a step and some time by just chaining them together with pipe to get the same result without generating a file:

ls | grep foo

Using piping, in two or three commands you can get an enormous amount of stuff done in a single line of code.