The Memory Bank

...for memory blanks

User Tools

Site Tools

Trace:
shell_redirection

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision		Previous revision
shell_redirection [2025/08/05 23:36] – created kenshell_redirection [2026/01/16 03:37] (current) – external edit 127.0.0.1
Line 3: Line 3:
 ======= 2&>1 ======= ======= 2&>1 =======
 \\ \\
-File descriptor 1 is the standard **output** (stdout).\\ +File descriptor **1** is the standard **output** (stdout).\\ 
-File descriptor 2 is the standard **error** (stderr).\\+File descriptor **2** is the standard **error** (stderr).\\
 \\ \\
 At first, **2>1** may look like a good way to redirect stderr to stdout. However, it will actually be interpreted as "redirect stderr to a file named 1".\\ At first, **2>1** may look like a good way to redirect stderr to stdout. However, it will actually be interpreted as "redirect stderr to a file named 1".\\
 "**&**" indicates that what follows and precedes is a file descriptor, and not a filename. Thus, we use **2>&1**. Consider **>&** to be a redirect merger operator. "**&**" indicates that what follows and precedes is a file descriptor, and not a filename. Thus, we use **2>&1**. Consider **>&** to be a redirect merger operator.
 \\ \\
 +==== To redirect stdout to file.txt: ====
 +\\
 +''echo test > file.txt''
 +\\
 +This is equivalent to:\\
 +\\
 +''echo test 1> file.txt''
 +\\
 +To redirect stderr to file.txt:\\
 +\\
 +''echo test 2> file.txt''
 +\\
 +So >& is the syntax to redirect a stream to another file descriptor:\\
 +\\
 +    0 is stdin\\
 +    1 is stdout\\
 +    2 is stderr\\
 +\\
 +To redirect stdout to stderr:\\
 +\\
 +''echo test 1>&2   # equivalently, echo test >&2''
 +\\
 +To redirect stderr to stdout:\\
 +\\
 +''echo test 2>&1''
 +\\
 +Thus, in 2>&1:\\
 +\\
 +    2> redirects stderr to an (unspecified) file.\\
 +    &1 redirects stderr to stdout.\\
 +\\
 +    @dbr cmd 2>&1 >>file does not redirect stderr to the file, but cmd >> file 2>&1 does. Order matters.\\
 +    In the first case, stderr is redirected to the stdout of the shell (possibly a tty if the command is entered interactively), \\
 +    and then stdout is directed to the file. In the second case, stdout is directed to the file, and then stderr is directed to the same place.\\
 +
 +\\
 +    I like the answer above, but it could be a touch clearer. "2>&1" redirects stderr to the target of stdout. \\
 +    So if you have something like "ls -l >> directoryContents 2>&1" The result will be a file named directoryContents \\
 +    will have the contents of the working directory appended to it. If there are any errors in execution: the error messages\\
 +    will also be appended to the directoryContents file, as they occur.\\
 +\\
 +    Is 0(or 1,2)>&0(or 1,2) like an option to control the output? Is echo test >test.log 2>&1 same as echo test 2>&1 >test.log?
 +    – Cloud
 +    Commented Jun 1, 2018 at 12:58
 +
 +    1
 +    for completeness (as top answer) - "redirect (and append) stdout and stderr to file and terminal and get proper exit status" command 2>&1 | tee -a file.txt
 +    – Estatistics
 +    Commented Aug 14, 2022 at 11:02
 +
 +
 +Some tricks about redirection
 +
 +Some syntax particularity about this may have important behaviours. There is some little samples about redirections, STDERR, STDOUT, and arguments ordering.
 +1 - Overwriting or appending?
 +
 +Symbol > means redirection.
 +
 +    > means send to as a whole completed file, overwriting target if exist (see noclobber bash feature at #3 later).
 +    >> means send in addition to would append to target if exist.
 +
 +In any case, the file would be created if they not exist.
 +2 - The shell command line is order dependent!!
 +
 +For testing this, we need a simple command which will send something on both outputs:
 +
 +$ ls -ld /tmp /tnt
 +ls: cannot access /tnt: No such file or directory
 +drwxrwxrwt 118 root root 196608 Jan  7 11:49 /tmp
 +
 +$ ls -ld /tmp /tnt >/dev/null
 +ls: cannot access /tnt: No such file or directory
 +
 +$ ls -ld /tmp /tnt 2>/dev/null
 +drwxrwxrwt 118 root root 196608 Jan  7 11:49 /tmp
 +
 +(Expecting you don't have a directory named /tnt, of course ;). Well, we have it!!
 +
 +So, let's see:
 +
 +$ ls -ld /tmp /tnt >/dev/null
 +ls: cannot access /tnt: No such file or directory
 +
 +$ ls -ld /tmp /tnt >/dev/null 2>&1
 +
 +$ ls -ld /tmp /tnt 2>&1 >/dev/null
 +ls: cannot access /tnt: No such file or directory
 +
 +The last command line dumps STDERR to the console, and it seem not to be the expected behaviour... But...
 +
 +If you want to make some post filtering about standard output, error output or both:
 +
 +$ ls -ld /tmp /tnt | sed 's/^.*$/<-- & --->/'
 +ls: cannot access /tnt: No such file or directory
 +<-- drwxrwxrwt 118 root root 196608 Jan  7 12:02 /tmp --->
 +
 +$ ls -ld /tmp /tnt 2>&1 | sed 's/^.*$/<-- & --->/'
 +<-- ls: cannot access /tnt: No such file or directory --->
 +<-- drwxrwxrwt 118 root root 196608 Jan  7 12:02 /tmp --->
 +
 +$ ls -ld /tmp /tnt >/dev/null | sed 's/^.*$/<-- & --->/'
 +ls: cannot access /tnt: No such file or directory
 +
 +$ ls -ld /tmp /tnt >/dev/null 2>&1 | sed 's/^.*$/<-- & --->/'
 +
 +$ ls -ld /tmp /tnt 2>&1 >/dev/null | sed 's/^.*$/<-- & --->/'
 +<-- ls: cannot access /tnt: No such file or directory --->
 +
 +Notice that the last command line in this paragraph is exactly same as in previous paragraph, where I wrote seem not to be the expected behaviour (so, this could even be an expected behaviour).
 +
 +Well, there is a little tricks about redirections, for doing different operation on both outputs:
 +
 +$ ( ls -ld /tmp /tnt | sed 's/^/O: /' >&9 ) 9>&2  2>&1  | sed 's/^/E: /'
 +O: drwxrwxrwt 118 root root 196608 Jan  7 12:13 /tmp
 +E: ls: cannot access /tnt: No such file or directory
 +
 +Note: &9 descriptor would occur spontaneously because of ) 9>&2.
 +
 +Addendum: nota! With the new version of bash (>4.0) there is a new feature and more sexy syntax for doing this kind of things:
 +
 +$ ls -ld /tmp /tnt 2> >(sed 's/^/E: /') > >(sed 's/^/O: /')
 +O: drwxrwxrwt 17 root root 28672 Nov  5 23:00 /tmp
 +E: ls: cannot access /tnt: No such file or directory
 +
 +And finally for such a cascading output formatting:
 +
 +$ ((ls -ld /tmp /tnt |sed 's/^/O: /' >&9 ) 2>&1 |sed 's/^/E: /') 9>&1| cat -n
 +     1  O: drwxrwxrwt 118 root root 196608 Jan  7 12:29 /tmp
 +     2  E: ls: cannot access /tnt: No such file or directory
 +
 +Addendum: nota! Same new syntax, in both ways:
 +
 +$ cat -n <(ls -ld /tmp /tnt 2> >(sed 's/^/E: /') > >(sed 's/^/O: /'))
 +     1  O: drwxrwxrwt 17 root root 28672 Nov  5 23:00 /tmp
 +     2  E: ls: cannot access /tnt: No such file or directory
 +
 +Where STDOUT go through a specific filter, STDERR to another and finally both outputs merged go through a third command filter.
 +2b - Using |& instead
 +
 +Syntax command |& ... could be used as an alias for command 2>&1 | .... Same rules about command line order applies. More details at What is the meaning of operator |& in bash?
 +3 - A word about noclobber option and >| syntax
 +
 +That's about overwriting:
 +
 +While set -o noclobber instruct bash to not overwrite any existing file, the >| syntax let you pass through this limitation:
 +
 +$ testfile=$(mktemp /tmp/testNoClobberDate-XXXXXX)
 +
 +$ date > $testfile ; cat $testfile
 +Mon Jan  7 13:18:15 CET 2013
 +
 +$ date > $testfile ; cat $testfile
 +Mon Jan  7 13:18:19 CET 2013
 +
 +$ date > $testfile ; cat $testfile
 +Mon Jan  7 13:18:21 CET 2013
 +
 +The file is overwritten each time, well now:
 +
 +$ set -o noclobber
 +
 +$ date > $testfile ; cat $testfile
 +bash: /tmp/testNoClobberDate-WW1xi9: cannot overwrite existing file
 +Mon Jan  7 13:18:21 CET 2013
 +
 +$ date > $testfile ; cat $testfile
 +bash: /tmp/testNoClobberDate-WW1xi9: cannot overwrite existing file
 +Mon Jan  7 13:18:21 CET 2013
 +
 +Pass through with >|:
 +
 +$ date >| $testfile ; cat $testfile
 +Mon Jan  7 13:18:58 CET 2013
 +
 +$ date >| $testfile ; cat $testfile
 +Mon Jan  7 13:19:01 CET 2013
 +
 +Unsetting this option and/or inquiring if already set.
 +
 +$ set -o | grep noclobber
 +noclobber           on
 +
 +$ set +o noclobber
 +
 +$ set -o | grep noclobber
 +noclobber           off
 +
 +$ date > $testfile ; cat $testfile
 +Mon Jan  7 13:24:27 CET 2013
 +
 +$ rm $testfile
 +
 +4 - Last trick and more...
 +
 +For redirecting both output from a given command, we see that a right syntax could be:
 +
 +$ ls -ld /tmp /tnt >/dev/null 2>&1
 +
 +for this special case, there is a shortcut syntax: &> ... or >&
 +
 +$ ls -ld /tmp /tnt &>/dev/null
 +
 +$ ls -ld /tmp /tnt >&/dev/null
 +
 +Nota: if 2>&1 exist, 1>&2 is a correct syntax too:
 +
 +$ ls -ld /tmp /tnt 2>/dev/null 1>&2
 +
 +4b- Now, I will let you think about:
 +
 +$ ls -ld /tmp /tnt 2>&1 1>&2  | sed -e s/^/++/
 +++/bin/ls: cannot access /tnt: No such file or directory
 +++drwxrwxrwt 193 root root 196608 Feb  9 11:08 /tmp/
 +
 +$ ls -ld /tmp /tnt 1>&2 2>&1  | sed -e s/^/++/
 +/bin/ls: cannot access /tnt: No such file or directory
 +drwxrwxrwt 193 root root 196608 Feb  9 11:08 /tmp/
 +
 +4c- If you're interested in more information
 +
 +You could read the fine manual by hitting:
 +
 +man -Len -Pless\ +/^REDIRECTION bash
 +
 +in a bash console ;-)
 +
 +I found this brilliant post on redirection: All about redirections
 +
 +Redirect both standard output and standard error to a file
 +
 +    $ command &>file
 +
 +This one-liner uses the &> operator to redirect both output streams - stdout and stderr - from command to file. This is Bash's shortcut for quickly redirecting both streams to the same destination.
 +
 +Here is how the file descriptor table looks like after Bash has redirected both streams:
 +
 +Enter image description here
 +
 +As you can see, both stdout and stderr now point to file. So anything written to stdout and stderr gets written to file.
 +
 +There are several ways to redirect both streams to the same destination. You can redirect each stream one after another:
 +
 +    $ command >file 2>&1
 +
 +This is a much more common way to redirect both streams to a file. First stdout is redirected to file, and then stderr is duplicated to be the same as stdout. So both streams end up pointing to file.
 +
 +When Bash sees several redirections it processes them from left to right. Let's go through the steps and see how that happens. Before running any commands, Bash's file descriptor table looks like this:
 +
 +Enter image description here
 +
 +Now Bash processes the first redirection >file. We've seen this before and it makes stdout point to file:
 +
 +Enter image description here
 +
 +Next Bash sees the second redirection 2>&1. We haven't seen this redirection before. This one duplicates file descriptor 2 to be a copy of file descriptor 1 and we get:
 +
 +Enter image description here
 +
 +Both streams have been redirected to file.
 +
 +However be careful here! Writing
 +
 +    command >file 2>&1
 +
 +is not the same as writing:
 +
 +    $ command 2>&1 >file
 +
 +The order of redirects matters in Bash! This command redirects only the standard output to the file. The stderr will still print to the terminal. To understand why that happens, let's go through the steps again. So before running the command, the file descriptor table looks like this:
 +
 +Enter image description here
 +
 +Now Bash processes redirections left to right. It first sees 2>&1 so it duplicates stderr to stdout. The file descriptor table becomes:
 +
 +Enter image description here
 +
 +Now Bash sees the second redirect, >file, and it redirects stdout to file:
 +
 +Enter image description here
 +
 +Do you see what happens here? Stdout now points to file, but the stderr still points to the terminal! Everything that gets written to stderr still gets printed out to the screen! So be very, very careful with the order of redirects!
 +
 +Also note that in Bash, writing
 +
 +    $ command &>file
 +
 +is exactly the same as:
 +
 +    $ command >&file
 +
 +
 +The numbers refer to the file descriptors (fd).
 +
 +    Zero is stdin
 +    One is stdout
 +    Two is stderr
 +
 +2>&1 redirects fd 2 to 1.
 +
 +This works for any number of file descriptors if the program uses them.
 +
 +You can look at /usr/include/unistd.h if you forget them:
 +
 +/* Standard file descriptors.  */
 +#define STDIN_FILENO    0   /* Standard input.  */
 +#define STDOUT_FILENO   1   /* Standard output.  */
 +#define STDERR_FILENO   2   /* Standard error output.  */
 +
 +That said I have written C tools that use non-standard file descriptors for custom logging so you don't see it unless you redirect it to a file or something.
 +
 +    Redirecting Input
 +
 +    Redirection of input causes the file whose name results from the expansion of word to be opened for reading on file descriptor n, or the standard input (file descriptor 0) if n is not specified.
 +
 +    The general format for redirecting input is:
 +
 +    [n]<word
 +
 +    Redirecting Output
 +
 +    Redirection of output causes the file whose name results from the expansion of word to be opened for writing on file descriptor n, or the standard output (file descriptor 1) if n is not specified. If the file does not exist it is created; if it does exist it is truncated to zero size.
 +
 +    The general format for redirecting output is:
 +
 +    [n]>word
 +
 +    Moving File Descriptors
 +
 +    The redirection operator,
 +
 +    [n]<&digit-
 +
 +    moves the file descriptor digit to file descriptor n, or the standard input (file descriptor 0) if n is not specified. digit is closed after being duplicated to n.
 +
 +    Similarly, the redirection operator
 +
 +    [n]>&digit-
 +
 +    moves the file descriptor digit to file descriptor n, or the standard output (file descriptor 1) if n is not specified.
 +
 +Ref:
 +
 +man bash
 +
 +Type /^REDIRECT to locate to the redirection section, and learn more...
 +
 +An online version is here: 3.6 Redirections
 +
shell_redirection.1754436995.txt.gz · Last modified: (external edit)

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki