Input / Output

Input/Output
Linux programs have a way of accepting input called standard input and a way of creating output called standard output.  In addition, each has a way of producing standard error output when there is a problem.  When a user sits down at the terminal they have standard input trough the use of a keyboard, standard output and standard error are sent to the computer screen.  

Lesson 7 / Lesson 9

stdin--This is short for "standard input".  By default, stdin comes from the keyboard.  However, by using pipes or redirectors, you can also obtain stdin from either a file, or from the output of another command.

stdout--This is short for "standard output".  By default, stdout gets sent to your computer screen.  You can use pipes to have stdout become the stdin for another command.  You can use redirectors to save the stdout as a file on disk.  Or, use redirectors to send stdout to the proverbial "bit bucket", so that you won't see any output at all.

stderr--As you may have guessed, this is short for "standard error".  If a command doesn't execute properly, you'll receive an error message.  By default, the message would show up on the screen.  However, you can use pipes or redirectors to change stderr's destination, just the same as you can with stdout.


stdin, stdout, and stderr are represented on a Linux system by file descriptors that are linked to from the /dev directory.  Each has an  ID number for the  file descriptors.

Name                                 ID Number
/dev/stdin                          0
/dev/stdout                       1
/dev/stderr                        2

Remembering these ID numbers will help you to understand concepts that are developed shortly.

A pipe will take the stdout of one command, and use it as stdin for another command.  It's represented by the “|” symbol.  (It's on the same key as the backslash.)

Piping stdout into another command as stdin

command | command2

To see how this can be useful, let's say that you want to look at a listing of all files in a certain directory.  But, there are so many files, that the output would scroll off of the screen where you'd never be able to see it.  You can solve the problem by taking the output (stdout) of the ls command, and using it as the input (stdin) for the less command.

       ls | less
   
Now, you'll be able to page through the entire output at your leisure.

With redirectors, you can either--

    * Direct a command to obtain its input (stdin) from someplace other than the keyboard.
    * Have a command send its output (stdout) to someplace other than the computer screen.
    * Have a command send its error messages (stderr) to someplace other than the computer screen.

There are several operator symbols that you need to keep track of with redirectors.  Look at the stdout (>, >|, and >>) family of operators. If wanted to look at the listing of files that are in a certain directory, iInstead of piping the ls output into less, you want to create a text file of this listing so that you can print it later.

        ls > filelist.txt

You can take pretty much any command that normally sends its output to the computer screen, and have it sent to a text file, instead.  There is one thing that you have to be careful with, though.  If you redirect a command's output to a file that already exists, you will overwrite it, and all previous information in that file will be lost. You can prevent this issue with an append.    

You can use the ">>" operator to append new information to the end of a file, instead of overwriting it.

          ls -la >> filelist.txt

There are file descriptors associated with stdin, stdout, and stderr.  Remember that each of these file descriptors has an ID number associated with it.  

You can use the file descriptor ID number for stdout when you create redirection commands.  This gives you an alternative.  Since the file descriptor ID for stdout is 1, you can make your redirection commands look like:

        ls -la 1> filelist.txt

        ls -la 1>| filelist.txt
or

        ls -la 1>> filelist.txt


stdin

There is only one operator symbol for stdin. By default, tr, a command that can translate, would take its stdin from the keyboard.  So, you could type a string in all lower-case, and have tr echo it back to you in all upper case.

     tr [:lower:] [:upper:]
        i only want to type in all lower-case letters.
        I ONLY WANT TO TYPE IN ALL LOWER-CASE LETTERS.
    
If you need for tr to take its input from a file, just add the appropriate redirector operator and the filename.

            tr [:lower:] [:upper:] < filelist.txt

This won't actually change the original file.  It will only cause the file contents to be displayed on screen in all upper-case letters.  If you want to save this converted output to another file, just add one of the stdout operators and a new filename.

    tr [:lower:] [:upper:] < filelist.txt > filelist_2.txt
   
Note:  When you use this trick, you'll always need to specify a new filename.  If you use this trick to try to write a new version of the original file, you'll end up with nothing but a file with no contents.  So, entering:

        tr [:lower:] [:upper:] < filelist.txt > filelist.txt

would be a bad thing.

Of course, you could also use the ">>" operator in this trick to append the new information to the original file.

    tr [:lower:] [:upper:] < testfile.txt >> testfile.txt
 

The stdin and stdout operators are fairly easy to understand.  The stderr operator isn't hard, but some aspects of it can be a bit tricky.

stderr

command 2> errorfile

The redirector operator for stderr is "2>".  If you're wondering why, it's because of the file descriptor ID numbers.

If you run a command and something goes wrong, it will output an error message (stderr).  By default, this message will get sent to the computer screen.  Also by default, stderr gets mixed in with stdout.  So, if your command outputs both good data and error messages, you'll have to scroll through the output messages on your screen to separate the two.  Fortunately, you can use redirectors to change that behavior.

If you're logged on to a computer as a normal user, and you use find to search through an entire file system, you'll get error messages when find tries to access directories that you don't have permission to look through.   You'll get good output, too.  But, note how the good is mixed in with the bad.

     find / -name README
    find: /boot/lost+found: Permission denied
    find: /etc/lvm/backup: Permission denied
    find: /etc/lvm/archive: Permission denied
---cut---

If you append a redirector and a filename to this command, you can send the error messages to a text file so that you can just look at the good data on the screen.

     find / -name README 2> errorfile
    /etc/httpd/conf.d/README
    /var/lib/xkb/README
    /var/www/icons/README
---cut---   

 
You can combine redirectors to have stdout sent to one text file, and stderr sent to another text file.

    find / -name README > results 2>  errorfile

If you don't want to see any error messages at all, just send stderr to the /dev/null device, known in some circles as the infamous "bit bucket".  

     find / -name README 2> /dev/null

If you want to send the good data to the "bit bucket", so that you'll see nothing but error messages, you can use this command.

     find / -name README > /dev/null

2>&1

Send both sterr and stdout to the same place.

If you want both stderr and stdout to go to the same text file, just enter your command with the regular stdout operator and destination.  Then, at the end, append "2>&1".  If you need a way to help understand this, just remember that stderr is File Descriptor ID 2, and stdout is File Descriptor ID 1.  So, you can read this as, "stderr (ID 2) goes to the same place as stdout (ID 1)".

To have both stderr and stdout of a find operation sent to the same text file, you can enter:

     find / -name README > results 2>&1

There may be times when you wouldn't want any output from either stderr or stdout.  If, for example, you needed to run a backup job in the background, you wouldn't want any screen output to mess up the text file that you're editing in the foreground.  (You also wouldn't need to save any of the output to a text file.)  For this, you can enter something like:

      find \( -iname '*.txt' -mtime -1 \) -exec cp {} /backup/ \; > /dev/null 2>&1