Redirection: how to save to a file the output from shell commands

The standard output

When a program returns some output to the terminal, it’s using a communication channel called standard output. For example, when we run ls, the output is printed on the terminal:

1

ls  -l ~/learn_bash/README.md  

will return:

1

-rw-r--r--  1 telatin  seqfu   58962 26 Oct 12:30 /home/telatin/learn_bash/README.md 

Redirecting the output

We can redirect the output of a program to a file, using the > character. For example, to save the output of ls to a file, we can do:

1

ls  -l ~/learn_bash/README.md  > ls_output.txt

If we hit “Enter”, we won’t see any output on the terminal, and we will get our prompt back. But if we look at the file ls_output.txt, we will see the output of ls.

The standard error

If we try this command:

1
2

# Here we feed to ls both an existing file and a non-existing file
ls -l ~/learn_bash-2022/phage/vir_genomic.fna /a/path/notfound.txt > ls_output_2.txt

We will see that the output of ls is saved in the file ls_output_2.txt, but we will also see an error message on the terminal:

1

ls: /a/path/notfound.txt: No such file or directory

How is this possible? Why are wee seeing some text if we did redirect the output? There are two communication channels in the terminal: the standard output and the standard error. The standard output is used to return the output of a program to the terminal. The standard error is used to return other messages (including errors).

If we want to redirect the standard error to a file, we can use the 2> character:

1
2

# Here we feed to ls both an existing file and a non-existing file
ls -l ~/learn_bash/README.md  /a/path/notfound.txt 2> ls_output_2.log

In this case we will see the README.md file listed, but the error is now silenced. This is because it was actually redirected to ls_output_2.log (check with cat!).

You can redirect both the standard output and the standard error to the same file, as summarised in this vignette:

Beware of overwriting files

If you redirect the output of a program to a file that already exists, the file will be overwritten. It’s less intutive but this also applies when you try to redirect the output from a file to the file itself. For example, if you try to run:

1

head longfile.txt  > longfile.txt

You would expect to see the first lines of the file longfile.txt saved in the same file, but because the redirection will occur before executing head, the resulting file would be empty!

Appending to a file

If you want to append the output of a program to an existing file, you can use the >> character, and this also applies to the standard error:

1
2
3

ls -l ~/learn_bash/README.md   > some_output.txt
ls -l ~/learn_bash/README.md   >> some_output.txt
ls -l ~/learn_bash/README.md   >> some_output.txt

The first command creates a new file called some_output.txt and saves the output of ls in it, the following commands will append the output of ls to the same file. The resulting file will contain three times the output of ls.

Similarily, if you want to redirect the standard error to a file, you can use the 2>> character:

1
2

ls -l ~/learn_bash/README.md  /a/path/notfound.txt 2>  ls_output_2.log
ls -l ~/learn_bash/README.md  /a/path/notfound.txt 2>> ls_output_2.log

Discarding the output

In linux everything is a file, meaning that in our filesystems we have files representing devices, processes, sockets, etc. There are special files representing our printer, or even the speakers of our computer.

A special device is called /dev/null, and it’s a file that discards everything that is written to it.

If you want to run a command but are not interested in the output, you can redirect it to /dev/null, or you can suppress the warnings/errors redirecting the standard error (2> /dev/null).