1. Home
  2. Docs
  3. Beginning UNIX on Raven
  4. Working with text files
  5. Piping and redirection

Piping and redirection

Piping is a very important concept in UNIX.   Put simply it allows you to send the output of a command/program directly into the input for another one.  This allows the creation of chains of commands which allow some quite powerful manipulation of data files without the need to resort to a programming based solution.

Linked to piping is the concept of redirection.  By default, the output of many commands is printed onto the screen.  By using redirection we can send this output to a file, either writing/overwriting a file or appending the output to an existing file.

Piping

Piping is achieved using the | syntax.  As an example we can string together 3 different commands to pull out the 2 classic lines from our American Pie lyrics.

cat americanpie.txt | tail -n 4 | head -n 2

Working through the logic of what we have done.  The command cat displays the file, which is then piped into tail to return the final 4 lines.  These lines are then piped into head which takes the top 2 lines (of the final 4 we already selected) and returns these to the display.

Redirection

If we want to save the results of our previous command to a file, we can just append a > character followed by our desired filename.

cat americanpie.txt | tail -n 4 | head -n 2 > junk.txt

Occasionally we might wish to redirect our output to file, but also have it displayed on screen as well to check the contents.  There are a number of ways to achieve this in a single line command.  Using the ; character, we can string together a number of lines of code which are executed sequentially:

head americanpie.txt > junk.txt; cat junk.txt

Alternatively (and more neatly), we can use the tee command to fork the output stream to both the display and a file:

americanpie.txt | tee junk.txt

Finally we may wish to append to data to a file.  For instance we can create an output log using the date command and the >> append syntax:

date > junk.txt

head americanpie.txt >> junk.txt

Summary of piping and redirection commands

| Pipe the output of one command to another
> Redirect the standard output to a file
>> Append the standard output to a file
tee Fork the output to both screen and a file