How do I delete lines of 1st file if it matches the string present in the 2nd file in linux?












1















Consider I have two text files.



First File name - "Emails.txt" with the following data:



00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00eeeeee@gotmail.com
00gggggg@uor.edu
00uuuuuuuu@yahoo.com
00e21_ss@cmail.com
00gggggggg@cmail.com
00zzzzzzzz48@hotmail.com
00aaaaaaa_2020@gotmail.com
jjjjjjjj@gmail.com


Second text file - "Banned.txt" with the following strings:



@gotmail.com
@cmail.com
@uor.edu


How to delete all the lines in the 1st text file "Emails.txt" if it matches the stings of any line present in the second text file "Banned.txt"?



The desired output of the new file should be:



00iiiiiiii_l@hotmail.com
00rrrrrrrr@hotmail.com
00zzzzz@gmail.com
00uuuuuuuu@yahoo.com
00zzzzzzzz48@hotmail.com
jjjjjjjj@gmail.com


Can this be done using SED or awk in Linux? Can you please suggest how to do this?










share|improve this question





























    1















    Consider I have two text files.



    First File name - "Emails.txt" with the following data:



    00iiiiiiii_l@hotmail.com
    00rrrrrrrr@hotmail.com
    00zzzzz@gmail.com
    00eeeeee@gotmail.com
    00gggggg@uor.edu
    00uuuuuuuu@yahoo.com
    00e21_ss@cmail.com
    00gggggggg@cmail.com
    00zzzzzzzz48@hotmail.com
    00aaaaaaa_2020@gotmail.com
    jjjjjjjj@gmail.com


    Second text file - "Banned.txt" with the following strings:



    @gotmail.com
    @cmail.com
    @uor.edu


    How to delete all the lines in the 1st text file "Emails.txt" if it matches the stings of any line present in the second text file "Banned.txt"?



    The desired output of the new file should be:



    00iiiiiiii_l@hotmail.com
    00rrrrrrrr@hotmail.com
    00zzzzz@gmail.com
    00uuuuuuuu@yahoo.com
    00zzzzzzzz48@hotmail.com
    jjjjjjjj@gmail.com


    Can this be done using SED or awk in Linux? Can you please suggest how to do this?










    share|improve this question



























      1












      1








      1








      Consider I have two text files.



      First File name - "Emails.txt" with the following data:



      00iiiiiiii_l@hotmail.com
      00rrrrrrrr@hotmail.com
      00zzzzz@gmail.com
      00eeeeee@gotmail.com
      00gggggg@uor.edu
      00uuuuuuuu@yahoo.com
      00e21_ss@cmail.com
      00gggggggg@cmail.com
      00zzzzzzzz48@hotmail.com
      00aaaaaaa_2020@gotmail.com
      jjjjjjjj@gmail.com


      Second text file - "Banned.txt" with the following strings:



      @gotmail.com
      @cmail.com
      @uor.edu


      How to delete all the lines in the 1st text file "Emails.txt" if it matches the stings of any line present in the second text file "Banned.txt"?



      The desired output of the new file should be:



      00iiiiiiii_l@hotmail.com
      00rrrrrrrr@hotmail.com
      00zzzzz@gmail.com
      00uuuuuuuu@yahoo.com
      00zzzzzzzz48@hotmail.com
      jjjjjjjj@gmail.com


      Can this be done using SED or awk in Linux? Can you please suggest how to do this?










      share|improve this question
















      Consider I have two text files.



      First File name - "Emails.txt" with the following data:



      00iiiiiiii_l@hotmail.com
      00rrrrrrrr@hotmail.com
      00zzzzz@gmail.com
      00eeeeee@gotmail.com
      00gggggg@uor.edu
      00uuuuuuuu@yahoo.com
      00e21_ss@cmail.com
      00gggggggg@cmail.com
      00zzzzzzzz48@hotmail.com
      00aaaaaaa_2020@gotmail.com
      jjjjjjjj@gmail.com


      Second text file - "Banned.txt" with the following strings:



      @gotmail.com
      @cmail.com
      @uor.edu


      How to delete all the lines in the 1st text file "Emails.txt" if it matches the stings of any line present in the second text file "Banned.txt"?



      The desired output of the new file should be:



      00iiiiiiii_l@hotmail.com
      00rrrrrrrr@hotmail.com
      00zzzzz@gmail.com
      00uuuuuuuu@yahoo.com
      00zzzzzzzz48@hotmail.com
      jjjjjjjj@gmail.com


      Can this be done using SED or awk in Linux? Can you please suggest how to do this?







      linux command-line sed






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jan 23 at 12:22









      grawity

      240k37508561




      240k37508561










      asked Jan 23 at 12:01









      Joney WalkerJoney Walker

      235




      235






















          1 Answer
          1






          active

          oldest

          votes


















          1














          grep -v is enough. The flag -f allows you to do exactly what you want:



          grep -vf Banned.txt Emails.txt


          If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'


          gives the desired



          @gotmail.com$|@cmail.com$|@uor.edu$


          Then:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt


          (doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.






          share|improve this answer


























          • This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt

            – Joney Walker
            Jan 23 at 12:46











          • That's the purpose of the -v. Have you tried it?

            – Joce
            Jan 23 at 12:48






          • 1





            grep -vf worked like a charm... Thanks.

            – Joney Walker
            Jan 23 at 12:58






          • 1





            After some more research, I found - Rather using a long regex, this works with this simple line too: grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt

            – Joney Walker
            Jan 28 at 12:04








          • 1





            @JoneyWalker Indeed that's shorter!

            – Joce
            Jan 28 at 15:27











          Your Answer








          StackExchange.ready(function() {
          var channelOptions = {
          tags: "".split(" "),
          id: "3"
          };
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function() {
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled) {
          StackExchange.using("snippets", function() {
          createEditor();
          });
          }
          else {
          createEditor();
          }
          });

          function createEditor() {
          StackExchange.prepareEditor({
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: true,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: 10,
          bindNavPrevention: true,
          postfix: "",
          imageUploader: {
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          },
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          });


          }
          });














          draft saved

          draft discarded


















          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1397436%2fhow-do-i-delete-lines-of-1st-file-if-it-matches-the-string-present-in-the-2nd-fi%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown

























          1 Answer
          1






          active

          oldest

          votes








          1 Answer
          1






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes









          1














          grep -v is enough. The flag -f allows you to do exactly what you want:



          grep -vf Banned.txt Emails.txt


          If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'


          gives the desired



          @gotmail.com$|@cmail.com$|@uor.edu$


          Then:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt


          (doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.






          share|improve this answer


























          • This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt

            – Joney Walker
            Jan 23 at 12:46











          • That's the purpose of the -v. Have you tried it?

            – Joce
            Jan 23 at 12:48






          • 1





            grep -vf worked like a charm... Thanks.

            – Joney Walker
            Jan 23 at 12:58






          • 1





            After some more research, I found - Rather using a long regex, this works with this simple line too: grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt

            – Joney Walker
            Jan 28 at 12:04








          • 1





            @JoneyWalker Indeed that's shorter!

            – Joce
            Jan 28 at 15:27
















          1














          grep -v is enough. The flag -f allows you to do exactly what you want:



          grep -vf Banned.txt Emails.txt


          If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'


          gives the desired



          @gotmail.com$|@cmail.com$|@uor.edu$


          Then:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt


          (doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.






          share|improve this answer


























          • This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt

            – Joney Walker
            Jan 23 at 12:46











          • That's the purpose of the -v. Have you tried it?

            – Joce
            Jan 23 at 12:48






          • 1





            grep -vf worked like a charm... Thanks.

            – Joney Walker
            Jan 23 at 12:58






          • 1





            After some more research, I found - Rather using a long regex, this works with this simple line too: grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt

            – Joney Walker
            Jan 28 at 12:04








          • 1





            @JoneyWalker Indeed that's shorter!

            – Joce
            Jan 28 at 15:27














          1












          1








          1







          grep -v is enough. The flag -f allows you to do exactly what you want:



          grep -vf Banned.txt Emails.txt


          If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'


          gives the desired



          @gotmail.com$|@cmail.com$|@uor.edu$


          Then:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt


          (doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.






          share|improve this answer















          grep -v is enough. The flag -f allows you to do exactly what you want:



          grep -vf Banned.txt Emails.txt


          If you want to do something more complicated out of the list of banned addresses, e.g. impose that they match the whole of the domain, you'll need to generate a regex from your Banned file:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\|,g' | sed -e 's,\|$,,'


          gives the desired



          @gotmail.com$|@cmail.com$|@uor.edu$


          Then:



          cat Banned.txt | tr "n" "|" | sed -e 's,|,$\\|,g' | sed -e 's,\|$,,' | xargs -i grep -v '{}' Emails.txt


          (doubling the number of escapes as they're being evaluated when going through xargs). This will match and remove me@uor.edu but not e.g. me@uor.education.gov.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Jan 24 at 14:15

























          answered Jan 23 at 12:34









          JoceJoce

          5001318




          5001318













          • This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt

            – Joney Walker
            Jan 23 at 12:46











          • That's the purpose of the -v. Have you tried it?

            – Joce
            Jan 23 at 12:48






          • 1





            grep -vf worked like a charm... Thanks.

            – Joney Walker
            Jan 23 at 12:58






          • 1





            After some more research, I found - Rather using a long regex, this works with this simple line too: grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt

            – Joney Walker
            Jan 28 at 12:04








          • 1





            @JoneyWalker Indeed that's shorter!

            – Joce
            Jan 28 at 15:27



















          • This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt

            – Joney Walker
            Jan 23 at 12:46











          • That's the purpose of the -v. Have you tried it?

            – Joce
            Jan 23 at 12:48






          • 1





            grep -vf worked like a charm... Thanks.

            – Joney Walker
            Jan 23 at 12:58






          • 1





            After some more research, I found - Rather using a long regex, this works with this simple line too: grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt

            – Joney Walker
            Jan 28 at 12:04








          • 1





            @JoneyWalker Indeed that's shorter!

            – Joce
            Jan 28 at 15:27

















          This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt

          – Joney Walker
          Jan 23 at 12:46





          This is not correct. The Output of the file should show ONLY the line whose sting doesn't match with Banned.txt file. Which means, all the domains / strings listed in the Banned.txt file should get deleted in the first file Emails.txt

          – Joney Walker
          Jan 23 at 12:46













          That's the purpose of the -v. Have you tried it?

          – Joce
          Jan 23 at 12:48





          That's the purpose of the -v. Have you tried it?

          – Joce
          Jan 23 at 12:48




          1




          1





          grep -vf worked like a charm... Thanks.

          – Joney Walker
          Jan 23 at 12:58





          grep -vf worked like a charm... Thanks.

          – Joney Walker
          Jan 23 at 12:58




          1




          1





          After some more research, I found - Rather using a long regex, this works with this simple line too: grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt

          – Joney Walker
          Jan 28 at 12:04







          After some more research, I found - Rather using a long regex, this works with this simple line too: grep -vf <(sed 's/$/$/' Banned.txt) Emails.txt

          – Joney Walker
          Jan 28 at 12:04






          1




          1





          @JoneyWalker Indeed that's shorter!

          – Joce
          Jan 28 at 15:27





          @JoneyWalker Indeed that's shorter!

          – Joce
          Jan 28 at 15:27


















          draft saved

          draft discarded




















































          Thanks for contributing an answer to Super User!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid



          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function () {
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1397436%2fhow-do-i-delete-lines-of-1st-file-if-it-matches-the-string-present-in-the-2nd-fi%23new-answer', 'question_page');
          }
          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          Plaza Victoria

          Puebla de Zaragoza

          Musa