what is the behavior of `>` `>` in shell












0















I know the basis of these three. And the reason why cat < file > file clears the content of file. But I don't understand why cat < file >> file would instead of writing the content twice, it concatenates the content like in an endless while loop.










share|improve this question



























    0















    I know the basis of these three. And the reason why cat < file > file clears the content of file. But I don't understand why cat < file >> file would instead of writing the content twice, it concatenates the content like in an endless while loop.










    share|improve this question

























      0












      0








      0








      I know the basis of these three. And the reason why cat < file > file clears the content of file. But I don't understand why cat < file >> file would instead of writing the content twice, it concatenates the content like in an endless while loop.










      share|improve this question














      I know the basis of these three. And the reason why cat < file > file clears the content of file. But I don't understand why cat < file >> file would instead of writing the content twice, it concatenates the content like in an endless while loop.







      shell






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Jan 13 at 1:56









      TiinaTiina

      6271614




      6271614






















          2 Answers
          2






          active

          oldest

          votes


















          1














          Well, my cat (in Kubuntu) says input file is output file and exits. I understand your cat is not as smart (or as lazy).



          What you observe is quite easy to explain. cat reads and writes in chunks. Suppose the first chunk doesn't reach the end of file (EOF) and its content gets concatenated at the end of the same file. Reading cat advanced by some amount of bytes but the end of the file "drifted away" by the same amount. In effect cat is again as far from EOF as it was at the beginning. And it goes on and on and on.



          As if you ran and for every 100 meters the finish line was moved additional 100 meters away from you. You would never reach it and the distance traveled would grow with no end.



          Unless the original distance is 100 meters or less. In this case you would get to the finish line before it's relocated. Analogously cat might finish if the file is small enough, but I really doubt it. I expect cat "knows" it got to EOF only when it tries to read again and is able to read 0 bytes, no more. This means it already read until EOF in the previous step, when it read more than 0 bytes. But in your case, if it read more than 0 bytes in the previous step, then the file grew immediately after and there is still data to read.



          I predict two cases so your cat will stop:




          • when the file is empty (0 bytes long),

          • when the file doesn't exist and you change the order of redirections (i.e. cat >> file < file will work, cat < file >> file will throw an error; this is because redirections are "served" from left to right, >> creates the file if needed, < requires the file to exist).




          In general using the same file with stdin and stdout (or/and stderr) redirection in a single command is asking for trouble. Still, if you need to do this, there is sponge utility (e.g. in moreutils package for Ubuntu, Debian).




          sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.



          If no output file is specified, sponge outputs to stdout.




          Instead of cat < file >> file run sponge < file >> file. It will do what you expected from cat (unless the file is huge enough to deplete your resources while being soaked up, I think).



          Another way is < file cat | cat >> file. Due to buffering between two cat-s the first one will really hit EOF and exit before the second one expands the file, if the file is small enough. Repeat this command few times starting with a small (but not empty) file and the file will eventually grow large enough to make the command behave like your original command with one cat. Note < file cat | unbuffer cat >> file gets rid of the buffer and loops even for very small files.






          share|improve this answer


























          • does this mean if I have a file that is endless and eventually larger than memory, no matter how slow data arrives, only if it does not write EOF, cat would handle it properly since it loads every trunk until EOF.

            – Tiina
            Jan 13 at 3:22











          • @Tiina If you have a data stream arriving slowly, cat will wait. Very simple to observe: run just cat, it will wait for your input. Type few lines, then it will wait again. Hit Ctrl+D to send a signal that will be interpreted as if EOF was reached.

            – Kamil Maciorowski
            Jan 13 at 3:43



















          0














          Well, basically this works as follows:




          > is used to override the content of a file. E.g: echo "HELLO WORLD" > file.txt.



          >> is used to append content to a file. E.g: echo "New line" >> file.txt.



          < you can use a file as input and redirect to another file. E.g: cat < file.txt > file2.txt. A practical example is ftp myftpbox.com < commands.txt, this would connect to a FTP and execute a list of commands place in the file commands.txt.




          NOTE: Be careful when you use > because you can override an existing file without any confirmation, you can alter this by running set -o noclobber.



          I hope this helps.






          share|improve this answer
























          • -1. Despite the OP says "I know the basis of these three", your answer gives the basis (well, set -o noclobber is beyond the basis, granted). What is more important, you didn't address the real issue at all.

            – Kamil Maciorowski
            Jan 13 at 3:30











          • @KamilMaciorowski +1 for the effor >.<

            – Tiina
            Jan 13 at 5:19











          • I guess I misunderstood last night. Your feedbacks are really appreciated here. Thanks

            – Manuel Florian
            Jan 13 at 9:40











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "3"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1393655%2fwhat-is-the-behavior-of-in-shell%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          2 Answers
          2






          active

          oldest

          votes








          2 Answers
          2






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          Well, my cat (in Kubuntu) says input file is output file and exits. I understand your cat is not as smart (or as lazy).



          What you observe is quite easy to explain. cat reads and writes in chunks. Suppose the first chunk doesn't reach the end of file (EOF) and its content gets concatenated at the end of the same file. Reading cat advanced by some amount of bytes but the end of the file "drifted away" by the same amount. In effect cat is again as far from EOF as it was at the beginning. And it goes on and on and on.



          As if you ran and for every 100 meters the finish line was moved additional 100 meters away from you. You would never reach it and the distance traveled would grow with no end.



          Unless the original distance is 100 meters or less. In this case you would get to the finish line before it's relocated. Analogously cat might finish if the file is small enough, but I really doubt it. I expect cat "knows" it got to EOF only when it tries to read again and is able to read 0 bytes, no more. This means it already read until EOF in the previous step, when it read more than 0 bytes. But in your case, if it read more than 0 bytes in the previous step, then the file grew immediately after and there is still data to read.



          I predict two cases so your cat will stop:




          • when the file is empty (0 bytes long),

          • when the file doesn't exist and you change the order of redirections (i.e. cat >> file < file will work, cat < file >> file will throw an error; this is because redirections are "served" from left to right, >> creates the file if needed, < requires the file to exist).




          In general using the same file with stdin and stdout (or/and stderr) redirection in a single command is asking for trouble. Still, if you need to do this, there is sponge utility (e.g. in moreutils package for Ubuntu, Debian).




          sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.



          If no output file is specified, sponge outputs to stdout.




          Instead of cat < file >> file run sponge < file >> file. It will do what you expected from cat (unless the file is huge enough to deplete your resources while being soaked up, I think).



          Another way is < file cat | cat >> file. Due to buffering between two cat-s the first one will really hit EOF and exit before the second one expands the file, if the file is small enough. Repeat this command few times starting with a small (but not empty) file and the file will eventually grow large enough to make the command behave like your original command with one cat. Note < file cat | unbuffer cat >> file gets rid of the buffer and loops even for very small files.






          share|improve this answer


























          • does this mean if I have a file that is endless and eventually larger than memory, no matter how slow data arrives, only if it does not write EOF, cat would handle it properly since it loads every trunk until EOF.

            – Tiina
            Jan 13 at 3:22











          • @Tiina If you have a data stream arriving slowly, cat will wait. Very simple to observe: run just cat, it will wait for your input. Type few lines, then it will wait again. Hit Ctrl+D to send a signal that will be interpreted as if EOF was reached.

            – Kamil Maciorowski
            Jan 13 at 3:43
















          1














          Well, my cat (in Kubuntu) says input file is output file and exits. I understand your cat is not as smart (or as lazy).



          What you observe is quite easy to explain. cat reads and writes in chunks. Suppose the first chunk doesn't reach the end of file (EOF) and its content gets concatenated at the end of the same file. Reading cat advanced by some amount of bytes but the end of the file "drifted away" by the same amount. In effect cat is again as far from EOF as it was at the beginning. And it goes on and on and on.



          As if you ran and for every 100 meters the finish line was moved additional 100 meters away from you. You would never reach it and the distance traveled would grow with no end.



          Unless the original distance is 100 meters or less. In this case you would get to the finish line before it's relocated. Analogously cat might finish if the file is small enough, but I really doubt it. I expect cat "knows" it got to EOF only when it tries to read again and is able to read 0 bytes, no more. This means it already read until EOF in the previous step, when it read more than 0 bytes. But in your case, if it read more than 0 bytes in the previous step, then the file grew immediately after and there is still data to read.



          I predict two cases so your cat will stop:




          • when the file is empty (0 bytes long),

          • when the file doesn't exist and you change the order of redirections (i.e. cat >> file < file will work, cat < file >> file will throw an error; this is because redirections are "served" from left to right, >> creates the file if needed, < requires the file to exist).




          In general using the same file with stdin and stdout (or/and stderr) redirection in a single command is asking for trouble. Still, if you need to do this, there is sponge utility (e.g. in moreutils package for Ubuntu, Debian).




          sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.



          If no output file is specified, sponge outputs to stdout.




          Instead of cat < file >> file run sponge < file >> file. It will do what you expected from cat (unless the file is huge enough to deplete your resources while being soaked up, I think).



          Another way is < file cat | cat >> file. Due to buffering between two cat-s the first one will really hit EOF and exit before the second one expands the file, if the file is small enough. Repeat this command few times starting with a small (but not empty) file and the file will eventually grow large enough to make the command behave like your original command with one cat. Note < file cat | unbuffer cat >> file gets rid of the buffer and loops even for very small files.






          share|improve this answer


























          • does this mean if I have a file that is endless and eventually larger than memory, no matter how slow data arrives, only if it does not write EOF, cat would handle it properly since it loads every trunk until EOF.

            – Tiina
            Jan 13 at 3:22











          • @Tiina If you have a data stream arriving slowly, cat will wait. Very simple to observe: run just cat, it will wait for your input. Type few lines, then it will wait again. Hit Ctrl+D to send a signal that will be interpreted as if EOF was reached.

            – Kamil Maciorowski
            Jan 13 at 3:43














          1












          1








          1







          Well, my cat (in Kubuntu) says input file is output file and exits. I understand your cat is not as smart (or as lazy).



          What you observe is quite easy to explain. cat reads and writes in chunks. Suppose the first chunk doesn't reach the end of file (EOF) and its content gets concatenated at the end of the same file. Reading cat advanced by some amount of bytes but the end of the file "drifted away" by the same amount. In effect cat is again as far from EOF as it was at the beginning. And it goes on and on and on.



          As if you ran and for every 100 meters the finish line was moved additional 100 meters away from you. You would never reach it and the distance traveled would grow with no end.



          Unless the original distance is 100 meters or less. In this case you would get to the finish line before it's relocated. Analogously cat might finish if the file is small enough, but I really doubt it. I expect cat "knows" it got to EOF only when it tries to read again and is able to read 0 bytes, no more. This means it already read until EOF in the previous step, when it read more than 0 bytes. But in your case, if it read more than 0 bytes in the previous step, then the file grew immediately after and there is still data to read.



          I predict two cases so your cat will stop:




          • when the file is empty (0 bytes long),

          • when the file doesn't exist and you change the order of redirections (i.e. cat >> file < file will work, cat < file >> file will throw an error; this is because redirections are "served" from left to right, >> creates the file if needed, < requires the file to exist).




          In general using the same file with stdin and stdout (or/and stderr) redirection in a single command is asking for trouble. Still, if you need to do this, there is sponge utility (e.g. in moreutils package for Ubuntu, Debian).




          sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.



          If no output file is specified, sponge outputs to stdout.




          Instead of cat < file >> file run sponge < file >> file. It will do what you expected from cat (unless the file is huge enough to deplete your resources while being soaked up, I think).



          Another way is < file cat | cat >> file. Due to buffering between two cat-s the first one will really hit EOF and exit before the second one expands the file, if the file is small enough. Repeat this command few times starting with a small (but not empty) file and the file will eventually grow large enough to make the command behave like your original command with one cat. Note < file cat | unbuffer cat >> file gets rid of the buffer and loops even for very small files.






          share|improve this answer















          Well, my cat (in Kubuntu) says input file is output file and exits. I understand your cat is not as smart (or as lazy).



          What you observe is quite easy to explain. cat reads and writes in chunks. Suppose the first chunk doesn't reach the end of file (EOF) and its content gets concatenated at the end of the same file. Reading cat advanced by some amount of bytes but the end of the file "drifted away" by the same amount. In effect cat is again as far from EOF as it was at the beginning. And it goes on and on and on.



          As if you ran and for every 100 meters the finish line was moved additional 100 meters away from you. You would never reach it and the distance traveled would grow with no end.



          Unless the original distance is 100 meters or less. In this case you would get to the finish line before it's relocated. Analogously cat might finish if the file is small enough, but I really doubt it. I expect cat "knows" it got to EOF only when it tries to read again and is able to read 0 bytes, no more. This means it already read until EOF in the previous step, when it read more than 0 bytes. But in your case, if it read more than 0 bytes in the previous step, then the file grew immediately after and there is still data to read.



          I predict two cases so your cat will stop:




          • when the file is empty (0 bytes long),

          • when the file doesn't exist and you change the order of redirections (i.e. cat >> file < file will work, cat < file >> file will throw an error; this is because redirections are "served" from left to right, >> creates the file if needed, < requires the file to exist).




          In general using the same file with stdin and stdout (or/and stderr) redirection in a single command is asking for trouble. Still, if you need to do this, there is sponge utility (e.g. in moreutils package for Ubuntu, Debian).




          sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before opening the output file. This allows constricting pipelines that read from and write to the same file.



          If no output file is specified, sponge outputs to stdout.




          Instead of cat < file >> file run sponge < file >> file. It will do what you expected from cat (unless the file is huge enough to deplete your resources while being soaked up, I think).



          Another way is < file cat | cat >> file. Due to buffering between two cat-s the first one will really hit EOF and exit before the second one expands the file, if the file is small enough. Repeat this command few times starting with a small (but not empty) file and the file will eventually grow large enough to make the command behave like your original command with one cat. Note < file cat | unbuffer cat >> file gets rid of the buffer and loops even for very small files.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Jan 13 at 3:53

























          answered Jan 13 at 3:05









          Kamil MaciorowskiKamil Maciorowski

          27.3k155982




          27.3k155982













          • does this mean if I have a file that is endless and eventually larger than memory, no matter how slow data arrives, only if it does not write EOF, cat would handle it properly since it loads every trunk until EOF.

            – Tiina
            Jan 13 at 3:22











          • @Tiina If you have a data stream arriving slowly, cat will wait. Very simple to observe: run just cat, it will wait for your input. Type few lines, then it will wait again. Hit Ctrl+D to send a signal that will be interpreted as if EOF was reached.

            – Kamil Maciorowski
            Jan 13 at 3:43



















          • does this mean if I have a file that is endless and eventually larger than memory, no matter how slow data arrives, only if it does not write EOF, cat would handle it properly since it loads every trunk until EOF.

            – Tiina
            Jan 13 at 3:22











          • @Tiina If you have a data stream arriving slowly, cat will wait. Very simple to observe: run just cat, it will wait for your input. Type few lines, then it will wait again. Hit Ctrl+D to send a signal that will be interpreted as if EOF was reached.

            – Kamil Maciorowski
            Jan 13 at 3:43

















          does this mean if I have a file that is endless and eventually larger than memory, no matter how slow data arrives, only if it does not write EOF, cat would handle it properly since it loads every trunk until EOF.

          – Tiina
          Jan 13 at 3:22





          does this mean if I have a file that is endless and eventually larger than memory, no matter how slow data arrives, only if it does not write EOF, cat would handle it properly since it loads every trunk until EOF.

          – Tiina
          Jan 13 at 3:22













          @Tiina If you have a data stream arriving slowly, cat will wait. Very simple to observe: run just cat, it will wait for your input. Type few lines, then it will wait again. Hit Ctrl+D to send a signal that will be interpreted as if EOF was reached.

          – Kamil Maciorowski
          Jan 13 at 3:43





          @Tiina If you have a data stream arriving slowly, cat will wait. Very simple to observe: run just cat, it will wait for your input. Type few lines, then it will wait again. Hit Ctrl+D to send a signal that will be interpreted as if EOF was reached.

          – Kamil Maciorowski
          Jan 13 at 3:43













          0














          Well, basically this works as follows:




          > is used to override the content of a file. E.g: echo "HELLO WORLD" > file.txt.



          >> is used to append content to a file. E.g: echo "New line" >> file.txt.



          < you can use a file as input and redirect to another file. E.g: cat < file.txt > file2.txt. A practical example is ftp myftpbox.com < commands.txt, this would connect to a FTP and execute a list of commands place in the file commands.txt.




          NOTE: Be careful when you use > because you can override an existing file without any confirmation, you can alter this by running set -o noclobber.



          I hope this helps.






          share|improve this answer
























          • -1. Despite the OP says "I know the basis of these three", your answer gives the basis (well, set -o noclobber is beyond the basis, granted). What is more important, you didn't address the real issue at all.

            – Kamil Maciorowski
            Jan 13 at 3:30











          • @KamilMaciorowski +1 for the effor >.<

            – Tiina
            Jan 13 at 5:19











          • I guess I misunderstood last night. Your feedbacks are really appreciated here. Thanks

            – Manuel Florian
            Jan 13 at 9:40
















          0














          Well, basically this works as follows:




          > is used to override the content of a file. E.g: echo "HELLO WORLD" > file.txt.



          >> is used to append content to a file. E.g: echo "New line" >> file.txt.



          < you can use a file as input and redirect to another file. E.g: cat < file.txt > file2.txt. A practical example is ftp myftpbox.com < commands.txt, this would connect to a FTP and execute a list of commands place in the file commands.txt.




          NOTE: Be careful when you use > because you can override an existing file without any confirmation, you can alter this by running set -o noclobber.



          I hope this helps.






          share|improve this answer
























          • -1. Despite the OP says "I know the basis of these three", your answer gives the basis (well, set -o noclobber is beyond the basis, granted). What is more important, you didn't address the real issue at all.

            – Kamil Maciorowski
            Jan 13 at 3:30











          • @KamilMaciorowski +1 for the effor >.<

            – Tiina
            Jan 13 at 5:19











          • I guess I misunderstood last night. Your feedbacks are really appreciated here. Thanks

            – Manuel Florian
            Jan 13 at 9:40














          0












          0








          0







          Well, basically this works as follows:




          > is used to override the content of a file. E.g: echo "HELLO WORLD" > file.txt.



          >> is used to append content to a file. E.g: echo "New line" >> file.txt.



          < you can use a file as input and redirect to another file. E.g: cat < file.txt > file2.txt. A practical example is ftp myftpbox.com < commands.txt, this would connect to a FTP and execute a list of commands place in the file commands.txt.




          NOTE: Be careful when you use > because you can override an existing file without any confirmation, you can alter this by running set -o noclobber.



          I hope this helps.






          share|improve this answer













          Well, basically this works as follows:




          > is used to override the content of a file. E.g: echo "HELLO WORLD" > file.txt.



          >> is used to append content to a file. E.g: echo "New line" >> file.txt.



          < you can use a file as input and redirect to another file. E.g: cat < file.txt > file2.txt. A practical example is ftp myftpbox.com < commands.txt, this would connect to a FTP and execute a list of commands place in the file commands.txt.




          NOTE: Be careful when you use > because you can override an existing file without any confirmation, you can alter this by running set -o noclobber.



          I hope this helps.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Jan 13 at 2:47









          Manuel FlorianManuel Florian

          1595




          1595













          • -1. Despite the OP says "I know the basis of these three", your answer gives the basis (well, set -o noclobber is beyond the basis, granted). What is more important, you didn't address the real issue at all.

            – Kamil Maciorowski
            Jan 13 at 3:30











          • @KamilMaciorowski +1 for the effor >.<

            – Tiina
            Jan 13 at 5:19











          • I guess I misunderstood last night. Your feedbacks are really appreciated here. Thanks

            – Manuel Florian
            Jan 13 at 9:40



















          • -1. Despite the OP says "I know the basis of these three", your answer gives the basis (well, set -o noclobber is beyond the basis, granted). What is more important, you didn't address the real issue at all.

            – Kamil Maciorowski
            Jan 13 at 3:30











          • @KamilMaciorowski +1 for the effor >.<

            – Tiina
            Jan 13 at 5:19











          • I guess I misunderstood last night. Your feedbacks are really appreciated here. Thanks

            – Manuel Florian
            Jan 13 at 9:40

















          -1. Despite the OP says "I know the basis of these three", your answer gives the basis (well, set -o noclobber is beyond the basis, granted). What is more important, you didn't address the real issue at all.

          – Kamil Maciorowski
          Jan 13 at 3:30





          -1. Despite the OP says "I know the basis of these three", your answer gives the basis (well, set -o noclobber is beyond the basis, granted). What is more important, you didn't address the real issue at all.

          – Kamil Maciorowski
          Jan 13 at 3:30













          @KamilMaciorowski +1 for the effor >.<

          – Tiina
          Jan 13 at 5:19





          @KamilMaciorowski +1 for the effor >.<

          – Tiina
          Jan 13 at 5:19













          I guess I misunderstood last night. Your feedbacks are really appreciated here. Thanks

          – Manuel Florian
          Jan 13 at 9:40





          I guess I misunderstood last night. Your feedbacks are really appreciated here. Thanks

          – Manuel Florian
          Jan 13 at 9:40


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Super User!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1393655%2fwhat-is-the-behavior-of-in-shell%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Plaza Victoria

          Brian Clough

          Cáceres