Delete empty column from csv file with bash script












2














Looking for some way to look at a csv file and delete columns including the header that have no values in the subsequent lines that follow.



Perhaps if I wanted to delete column Test03 from below including Test03 in the first line.



Test01,Test02,Test03,Test04  
11,22,,44
11,22,,44
11,22,,44
11,22,,44
11,22,,44
11,22,,44









share|improve this question
























  • The third column has a value in the first row but not the rest. Do you want to delete the third column in all but the first row?
    – John1024
    May 7 '14 at 6:18










  • I can help you with an awk command for this operation, but tell me this: Will the second line always be representative of the rest of the following lines?
    – bgStack15
    May 7 '14 at 15:43










  • Yes the second line and subsequent lines will have no values in column 3.
    – user1988900
    May 7 '14 at 16:08
















2














Looking for some way to look at a csv file and delete columns including the header that have no values in the subsequent lines that follow.



Perhaps if I wanted to delete column Test03 from below including Test03 in the first line.



Test01,Test02,Test03,Test04  
11,22,,44
11,22,,44
11,22,,44
11,22,,44
11,22,,44
11,22,,44









share|improve this question
























  • The third column has a value in the first row but not the rest. Do you want to delete the third column in all but the first row?
    – John1024
    May 7 '14 at 6:18










  • I can help you with an awk command for this operation, but tell me this: Will the second line always be representative of the rest of the following lines?
    – bgStack15
    May 7 '14 at 15:43










  • Yes the second line and subsequent lines will have no values in column 3.
    – user1988900
    May 7 '14 at 16:08














2












2








2


0





Looking for some way to look at a csv file and delete columns including the header that have no values in the subsequent lines that follow.



Perhaps if I wanted to delete column Test03 from below including Test03 in the first line.



Test01,Test02,Test03,Test04  
11,22,,44
11,22,,44
11,22,,44
11,22,,44
11,22,,44
11,22,,44









share|improve this question















Looking for some way to look at a csv file and delete columns including the header that have no values in the subsequent lines that follow.



Perhaps if I wanted to delete column Test03 from below including Test03 in the first line.



Test01,Test02,Test03,Test04  
11,22,,44
11,22,,44
11,22,,44
11,22,,44
11,22,,44
11,22,,44






linux bash csv






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited May 7 '14 at 14:06

























asked May 7 '14 at 6:05









user1988900

1113




1113












  • The third column has a value in the first row but not the rest. Do you want to delete the third column in all but the first row?
    – John1024
    May 7 '14 at 6:18










  • I can help you with an awk command for this operation, but tell me this: Will the second line always be representative of the rest of the following lines?
    – bgStack15
    May 7 '14 at 15:43










  • Yes the second line and subsequent lines will have no values in column 3.
    – user1988900
    May 7 '14 at 16:08


















  • The third column has a value in the first row but not the rest. Do you want to delete the third column in all but the first row?
    – John1024
    May 7 '14 at 6:18










  • I can help you with an awk command for this operation, but tell me this: Will the second line always be representative of the rest of the following lines?
    – bgStack15
    May 7 '14 at 15:43










  • Yes the second line and subsequent lines will have no values in column 3.
    – user1988900
    May 7 '14 at 16:08
















The third column has a value in the first row but not the rest. Do you want to delete the third column in all but the first row?
– John1024
May 7 '14 at 6:18




The third column has a value in the first row but not the rest. Do you want to delete the third column in all but the first row?
– John1024
May 7 '14 at 6:18












I can help you with an awk command for this operation, but tell me this: Will the second line always be representative of the rest of the following lines?
– bgStack15
May 7 '14 at 15:43




I can help you with an awk command for this operation, but tell me this: Will the second line always be representative of the rest of the following lines?
– bgStack15
May 7 '14 at 15:43












Yes the second line and subsequent lines will have no values in column 3.
– user1988900
May 7 '14 at 16:08




Yes the second line and subsequent lines will have no values in column 3.
– user1988900
May 7 '14 at 16:08










8 Answers
8






active

oldest

votes


















1














Here's an awk solution that performs agnostic to whichever columns are empty (ignoring the header).



awk -F, '{
a[NR]=$0
}NR>1{
for (i=1;i<=NF;i++)
if(length($i)!=0) b[i]++
}END{
for (k=1;k<=NR;k++) {
LINE="" ;
split(a[k],c,",") ;
for (j=1;j<=NF;j++)
if(b[j]>0)
LINE=LINE","c[j] ;
print substr(LINE,2,length(LINE)-1)
}
}' test.csv





share|improve this answer





















  • I'm not a big fan of the style of starting the next stanza on the same line as the } of the previous one, but, otherwise, this looks like a good answer.  Welcome to Super User, and good job!  I hope you continue to make contributions as good as this.
    – Scott
    Dec 1 at 15:48



















0














In current case you can just do:



sed 's/,,/,/g' test.csv > new.csv


This'll replace all double commas with just one, effectively removing your empty column. Note that you'll need to remove the column from the header yourself.






share|improve this answer





















  • Yes that will remove the empty column from lines after the first but I want to remove the column entirely if there are no values but in the first automatically.
    – user1988900
    May 7 '14 at 14:08



















0














If you want to delete possibly non-empty columns (including in the header), use the 'cut' command:



cut -d , -f 1,2,4 test.csv > new.csv





share|improve this answer





















  • This is fine for plain numeric data, but beware that CSV values can contain commas (when properly quoted) and this will not handle that - you would need a full CSV parser.
    – Andrew Medico
    May 7 '14 at 17:44










  • Yes aware of cut and it's great if you know which columns are empty but for this I don't always know. Andrew - which CSV parser would you refer to?
    – user1988900
    May 7 '14 at 19:46



















0















  • If repeated empty columns ,,,.

  • If whitespaces columns 1, , ,,2 (start with/in middle/at the end).

  • If it's empty at first or end of a line ,123/123,



sed -E ':l;s/,[[:blank:]]*,/,/;tl;s/^[[:blank:]]*,|,[[:blank:]]*$//g'





share|improve this answer





























    0














    awk joins the party.



    awk -F "," '{print $1","$2","$4}' test.csv > new.csv





    share|improve this answer





























      0














      This calls for a program rather than a quick command. The best way to do it would be, as suggested by Andrew Medico, to employ a proper CSV parser (in the case of perl you have Text::CSV).



      However, I thought I'd write a perl script that works in very simple cases:



      perl -F, -lane 'if($.==1){@a=@F;next};for($i=0;$i<@F;$i++){if($F[$i]!=""){push @c,$F[$i];push @b,$i}}if(@a){foreach(@b){push @t,$a[$_]};print join(",",@t);undef @a}print join(",",@c);undef @c' file.csv



      This saves the first line and goes on to see if there are any empty fields in the next line. It then prints only the relevant headers, skipping the empty field in all lines.



      Please note that it doesn't handle commas inside quoted strings. It does, however, turn:



      Test01,Test02,Test03,Test04
      11,22,,44
      11,22,,44
      11,22,,44
      11,22,,44
      11,22,,44
      11,22,,44


      into:



      Test01,Test02,Test04
      11,22,44
      11,22,44
      11,22,44
      11,22,44
      11,22,44
      11,22,44





      share|improve this answer





























        0














        While trying different bash approaches I needed to remove all empty columns (including the header) reliably. To solve this I used Python with Pandas.



        import pandas as pd

        data = pd.read_csv('test.csv', sep='t')
        data.dropna(axis=1).to_csv('test_clean.csv')


        The important thing here is to add the axis=1 to tell Pandas to apply the dropna to columns instead of rows.






        share|improve this answer





























          0














          For a typical user the easiest way would be to import data in Excel from this CSV file and export it once again after removing the column.






          share|improve this answer





















            Your Answer








            StackExchange.ready(function() {
            var channelOptions = {
            tags: "".split(" "),
            id: "3"
            };
            initTagRenderer("".split(" "), "".split(" "), channelOptions);

            StackExchange.using("externalEditor", function() {
            // Have to fire editor after snippets, if snippets enabled
            if (StackExchange.settings.snippets.snippetsEnabled) {
            StackExchange.using("snippets", function() {
            createEditor();
            });
            }
            else {
            createEditor();
            }
            });

            function createEditor() {
            StackExchange.prepareEditor({
            heartbeatType: 'answer',
            autoActivateHeartbeat: false,
            convertImagesToLinks: true,
            noModals: true,
            showLowRepImageUploadWarning: true,
            reputationToPostImages: 10,
            bindNavPrevention: true,
            postfix: "",
            imageUploader: {
            brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
            contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
            allowUrls: true
            },
            onDemand: true,
            discardSelector: ".discard-answer"
            ,immediatelyShowMarkdownHelp:true
            });


            }
            });














            draft saved

            draft discarded


















            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f750651%2fdelete-empty-column-from-csv-file-with-bash-script%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown

























            8 Answers
            8






            active

            oldest

            votes








            8 Answers
            8






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes









            1














            Here's an awk solution that performs agnostic to whichever columns are empty (ignoring the header).



            awk -F, '{
            a[NR]=$0
            }NR>1{
            for (i=1;i<=NF;i++)
            if(length($i)!=0) b[i]++
            }END{
            for (k=1;k<=NR;k++) {
            LINE="" ;
            split(a[k],c,",") ;
            for (j=1;j<=NF;j++)
            if(b[j]>0)
            LINE=LINE","c[j] ;
            print substr(LINE,2,length(LINE)-1)
            }
            }' test.csv





            share|improve this answer





















            • I'm not a big fan of the style of starting the next stanza on the same line as the } of the previous one, but, otherwise, this looks like a good answer.  Welcome to Super User, and good job!  I hope you continue to make contributions as good as this.
              – Scott
              Dec 1 at 15:48
















            1














            Here's an awk solution that performs agnostic to whichever columns are empty (ignoring the header).



            awk -F, '{
            a[NR]=$0
            }NR>1{
            for (i=1;i<=NF;i++)
            if(length($i)!=0) b[i]++
            }END{
            for (k=1;k<=NR;k++) {
            LINE="" ;
            split(a[k],c,",") ;
            for (j=1;j<=NF;j++)
            if(b[j]>0)
            LINE=LINE","c[j] ;
            print substr(LINE,2,length(LINE)-1)
            }
            }' test.csv





            share|improve this answer





















            • I'm not a big fan of the style of starting the next stanza on the same line as the } of the previous one, but, otherwise, this looks like a good answer.  Welcome to Super User, and good job!  I hope you continue to make contributions as good as this.
              – Scott
              Dec 1 at 15:48














            1












            1








            1






            Here's an awk solution that performs agnostic to whichever columns are empty (ignoring the header).



            awk -F, '{
            a[NR]=$0
            }NR>1{
            for (i=1;i<=NF;i++)
            if(length($i)!=0) b[i]++
            }END{
            for (k=1;k<=NR;k++) {
            LINE="" ;
            split(a[k],c,",") ;
            for (j=1;j<=NF;j++)
            if(b[j]>0)
            LINE=LINE","c[j] ;
            print substr(LINE,2,length(LINE)-1)
            }
            }' test.csv





            share|improve this answer












            Here's an awk solution that performs agnostic to whichever columns are empty (ignoring the header).



            awk -F, '{
            a[NR]=$0
            }NR>1{
            for (i=1;i<=NF;i++)
            if(length($i)!=0) b[i]++
            }END{
            for (k=1;k<=NR;k++) {
            LINE="" ;
            split(a[k],c,",") ;
            for (j=1;j<=NF;j++)
            if(b[j]>0)
            LINE=LINE","c[j] ;
            print substr(LINE,2,length(LINE)-1)
            }
            }' test.csv






            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Dec 1 at 14:53









            user2606364

            111




            111












            • I'm not a big fan of the style of starting the next stanza on the same line as the } of the previous one, but, otherwise, this looks like a good answer.  Welcome to Super User, and good job!  I hope you continue to make contributions as good as this.
              – Scott
              Dec 1 at 15:48


















            • I'm not a big fan of the style of starting the next stanza on the same line as the } of the previous one, but, otherwise, this looks like a good answer.  Welcome to Super User, and good job!  I hope you continue to make contributions as good as this.
              – Scott
              Dec 1 at 15:48
















            I'm not a big fan of the style of starting the next stanza on the same line as the } of the previous one, but, otherwise, this looks like a good answer.  Welcome to Super User, and good job!  I hope you continue to make contributions as good as this.
            – Scott
            Dec 1 at 15:48




            I'm not a big fan of the style of starting the next stanza on the same line as the } of the previous one, but, otherwise, this looks like a good answer.  Welcome to Super User, and good job!  I hope you continue to make contributions as good as this.
            – Scott
            Dec 1 at 15:48













            0














            In current case you can just do:



            sed 's/,,/,/g' test.csv > new.csv


            This'll replace all double commas with just one, effectively removing your empty column. Note that you'll need to remove the column from the header yourself.






            share|improve this answer





















            • Yes that will remove the empty column from lines after the first but I want to remove the column entirely if there are no values but in the first automatically.
              – user1988900
              May 7 '14 at 14:08
















            0














            In current case you can just do:



            sed 's/,,/,/g' test.csv > new.csv


            This'll replace all double commas with just one, effectively removing your empty column. Note that you'll need to remove the column from the header yourself.






            share|improve this answer





















            • Yes that will remove the empty column from lines after the first but I want to remove the column entirely if there are no values but in the first automatically.
              – user1988900
              May 7 '14 at 14:08














            0












            0








            0






            In current case you can just do:



            sed 's/,,/,/g' test.csv > new.csv


            This'll replace all double commas with just one, effectively removing your empty column. Note that you'll need to remove the column from the header yourself.






            share|improve this answer












            In current case you can just do:



            sed 's/,,/,/g' test.csv > new.csv


            This'll replace all double commas with just one, effectively removing your empty column. Note that you'll need to remove the column from the header yourself.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered May 7 '14 at 8:12









            Priit

            1092




            1092












            • Yes that will remove the empty column from lines after the first but I want to remove the column entirely if there are no values but in the first automatically.
              – user1988900
              May 7 '14 at 14:08


















            • Yes that will remove the empty column from lines after the first but I want to remove the column entirely if there are no values but in the first automatically.
              – user1988900
              May 7 '14 at 14:08
















            Yes that will remove the empty column from lines after the first but I want to remove the column entirely if there are no values but in the first automatically.
            – user1988900
            May 7 '14 at 14:08




            Yes that will remove the empty column from lines after the first but I want to remove the column entirely if there are no values but in the first automatically.
            – user1988900
            May 7 '14 at 14:08











            0














            If you want to delete possibly non-empty columns (including in the header), use the 'cut' command:



            cut -d , -f 1,2,4 test.csv > new.csv





            share|improve this answer





















            • This is fine for plain numeric data, but beware that CSV values can contain commas (when properly quoted) and this will not handle that - you would need a full CSV parser.
              – Andrew Medico
              May 7 '14 at 17:44










            • Yes aware of cut and it's great if you know which columns are empty but for this I don't always know. Andrew - which CSV parser would you refer to?
              – user1988900
              May 7 '14 at 19:46
















            0














            If you want to delete possibly non-empty columns (including in the header), use the 'cut' command:



            cut -d , -f 1,2,4 test.csv > new.csv





            share|improve this answer





















            • This is fine for plain numeric data, but beware that CSV values can contain commas (when properly quoted) and this will not handle that - you would need a full CSV parser.
              – Andrew Medico
              May 7 '14 at 17:44










            • Yes aware of cut and it's great if you know which columns are empty but for this I don't always know. Andrew - which CSV parser would you refer to?
              – user1988900
              May 7 '14 at 19:46














            0












            0








            0






            If you want to delete possibly non-empty columns (including in the header), use the 'cut' command:



            cut -d , -f 1,2,4 test.csv > new.csv





            share|improve this answer












            If you want to delete possibly non-empty columns (including in the header), use the 'cut' command:



            cut -d , -f 1,2,4 test.csv > new.csv






            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered May 7 '14 at 17:24









            gogators

            1,103612




            1,103612












            • This is fine for plain numeric data, but beware that CSV values can contain commas (when properly quoted) and this will not handle that - you would need a full CSV parser.
              – Andrew Medico
              May 7 '14 at 17:44










            • Yes aware of cut and it's great if you know which columns are empty but for this I don't always know. Andrew - which CSV parser would you refer to?
              – user1988900
              May 7 '14 at 19:46


















            • This is fine for plain numeric data, but beware that CSV values can contain commas (when properly quoted) and this will not handle that - you would need a full CSV parser.
              – Andrew Medico
              May 7 '14 at 17:44










            • Yes aware of cut and it's great if you know which columns are empty but for this I don't always know. Andrew - which CSV parser would you refer to?
              – user1988900
              May 7 '14 at 19:46
















            This is fine for plain numeric data, but beware that CSV values can contain commas (when properly quoted) and this will not handle that - you would need a full CSV parser.
            – Andrew Medico
            May 7 '14 at 17:44




            This is fine for plain numeric data, but beware that CSV values can contain commas (when properly quoted) and this will not handle that - you would need a full CSV parser.
            – Andrew Medico
            May 7 '14 at 17:44












            Yes aware of cut and it's great if you know which columns are empty but for this I don't always know. Andrew - which CSV parser would you refer to?
            – user1988900
            May 7 '14 at 19:46




            Yes aware of cut and it's great if you know which columns are empty but for this I don't always know. Andrew - which CSV parser would you refer to?
            – user1988900
            May 7 '14 at 19:46











            0















            • If repeated empty columns ,,,.

            • If whitespaces columns 1, , ,,2 (start with/in middle/at the end).

            • If it's empty at first or end of a line ,123/123,



            sed -E ':l;s/,[[:blank:]]*,/,/;tl;s/^[[:blank:]]*,|,[[:blank:]]*$//g'





            share|improve this answer


























              0















              • If repeated empty columns ,,,.

              • If whitespaces columns 1, , ,,2 (start with/in middle/at the end).

              • If it's empty at first or end of a line ,123/123,



              sed -E ':l;s/,[[:blank:]]*,/,/;tl;s/^[[:blank:]]*,|,[[:blank:]]*$//g'





              share|improve this answer
























                0












                0








                0







                • If repeated empty columns ,,,.

                • If whitespaces columns 1, , ,,2 (start with/in middle/at the end).

                • If it's empty at first or end of a line ,123/123,



                sed -E ':l;s/,[[:blank:]]*,/,/;tl;s/^[[:blank:]]*,|,[[:blank:]]*$//g'





                share|improve this answer













                • If repeated empty columns ,,,.

                • If whitespaces columns 1, , ,,2 (start with/in middle/at the end).

                • If it's empty at first or end of a line ,123/123,



                sed -E ':l;s/,[[:blank:]]*,/,/;tl;s/^[[:blank:]]*,|,[[:blank:]]*$//g'






                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Oct 20 '17 at 7:55









                αғsнιη

                3841217




                3841217























                    0














                    awk joins the party.



                    awk -F "," '{print $1","$2","$4}' test.csv > new.csv





                    share|improve this answer


























                      0














                      awk joins the party.



                      awk -F "," '{print $1","$2","$4}' test.csv > new.csv





                      share|improve this answer
























                        0












                        0








                        0






                        awk joins the party.



                        awk -F "," '{print $1","$2","$4}' test.csv > new.csv





                        share|improve this answer












                        awk joins the party.



                        awk -F "," '{print $1","$2","$4}' test.csv > new.csv






                        share|improve this answer












                        share|improve this answer



                        share|improve this answer










                        answered Oct 20 '17 at 9:29









                        chingNotCHing

                        611311




                        611311























                            0














                            This calls for a program rather than a quick command. The best way to do it would be, as suggested by Andrew Medico, to employ a proper CSV parser (in the case of perl you have Text::CSV).



                            However, I thought I'd write a perl script that works in very simple cases:



                            perl -F, -lane 'if($.==1){@a=@F;next};for($i=0;$i<@F;$i++){if($F[$i]!=""){push @c,$F[$i];push @b,$i}}if(@a){foreach(@b){push @t,$a[$_]};print join(",",@t);undef @a}print join(",",@c);undef @c' file.csv



                            This saves the first line and goes on to see if there are any empty fields in the next line. It then prints only the relevant headers, skipping the empty field in all lines.



                            Please note that it doesn't handle commas inside quoted strings. It does, however, turn:



                            Test01,Test02,Test03,Test04
                            11,22,,44
                            11,22,,44
                            11,22,,44
                            11,22,,44
                            11,22,,44
                            11,22,,44


                            into:



                            Test01,Test02,Test04
                            11,22,44
                            11,22,44
                            11,22,44
                            11,22,44
                            11,22,44
                            11,22,44





                            share|improve this answer


























                              0














                              This calls for a program rather than a quick command. The best way to do it would be, as suggested by Andrew Medico, to employ a proper CSV parser (in the case of perl you have Text::CSV).



                              However, I thought I'd write a perl script that works in very simple cases:



                              perl -F, -lane 'if($.==1){@a=@F;next};for($i=0;$i<@F;$i++){if($F[$i]!=""){push @c,$F[$i];push @b,$i}}if(@a){foreach(@b){push @t,$a[$_]};print join(",",@t);undef @a}print join(",",@c);undef @c' file.csv



                              This saves the first line and goes on to see if there are any empty fields in the next line. It then prints only the relevant headers, skipping the empty field in all lines.



                              Please note that it doesn't handle commas inside quoted strings. It does, however, turn:



                              Test01,Test02,Test03,Test04
                              11,22,,44
                              11,22,,44
                              11,22,,44
                              11,22,,44
                              11,22,,44
                              11,22,,44


                              into:



                              Test01,Test02,Test04
                              11,22,44
                              11,22,44
                              11,22,44
                              11,22,44
                              11,22,44
                              11,22,44





                              share|improve this answer
























                                0












                                0








                                0






                                This calls for a program rather than a quick command. The best way to do it would be, as suggested by Andrew Medico, to employ a proper CSV parser (in the case of perl you have Text::CSV).



                                However, I thought I'd write a perl script that works in very simple cases:



                                perl -F, -lane 'if($.==1){@a=@F;next};for($i=0;$i<@F;$i++){if($F[$i]!=""){push @c,$F[$i];push @b,$i}}if(@a){foreach(@b){push @t,$a[$_]};print join(",",@t);undef @a}print join(",",@c);undef @c' file.csv



                                This saves the first line and goes on to see if there are any empty fields in the next line. It then prints only the relevant headers, skipping the empty field in all lines.



                                Please note that it doesn't handle commas inside quoted strings. It does, however, turn:



                                Test01,Test02,Test03,Test04
                                11,22,,44
                                11,22,,44
                                11,22,,44
                                11,22,,44
                                11,22,,44
                                11,22,,44


                                into:



                                Test01,Test02,Test04
                                11,22,44
                                11,22,44
                                11,22,44
                                11,22,44
                                11,22,44
                                11,22,44





                                share|improve this answer












                                This calls for a program rather than a quick command. The best way to do it would be, as suggested by Andrew Medico, to employ a proper CSV parser (in the case of perl you have Text::CSV).



                                However, I thought I'd write a perl script that works in very simple cases:



                                perl -F, -lane 'if($.==1){@a=@F;next};for($i=0;$i<@F;$i++){if($F[$i]!=""){push @c,$F[$i];push @b,$i}}if(@a){foreach(@b){push @t,$a[$_]};print join(",",@t);undef @a}print join(",",@c);undef @c' file.csv



                                This saves the first line and goes on to see if there are any empty fields in the next line. It then prints only the relevant headers, skipping the empty field in all lines.



                                Please note that it doesn't handle commas inside quoted strings. It does, however, turn:



                                Test01,Test02,Test03,Test04
                                11,22,,44
                                11,22,,44
                                11,22,,44
                                11,22,,44
                                11,22,,44
                                11,22,,44


                                into:



                                Test01,Test02,Test04
                                11,22,44
                                11,22,44
                                11,22,44
                                11,22,44
                                11,22,44
                                11,22,44






                                share|improve this answer












                                share|improve this answer



                                share|improve this answer










                                answered Oct 27 '17 at 11:11









                                simlev

                                3,0163527




                                3,0163527























                                    0














                                    While trying different bash approaches I needed to remove all empty columns (including the header) reliably. To solve this I used Python with Pandas.



                                    import pandas as pd

                                    data = pd.read_csv('test.csv', sep='t')
                                    data.dropna(axis=1).to_csv('test_clean.csv')


                                    The important thing here is to add the axis=1 to tell Pandas to apply the dropna to columns instead of rows.






                                    share|improve this answer


























                                      0














                                      While trying different bash approaches I needed to remove all empty columns (including the header) reliably. To solve this I used Python with Pandas.



                                      import pandas as pd

                                      data = pd.read_csv('test.csv', sep='t')
                                      data.dropna(axis=1).to_csv('test_clean.csv')


                                      The important thing here is to add the axis=1 to tell Pandas to apply the dropna to columns instead of rows.






                                      share|improve this answer
























                                        0












                                        0








                                        0






                                        While trying different bash approaches I needed to remove all empty columns (including the header) reliably. To solve this I used Python with Pandas.



                                        import pandas as pd

                                        data = pd.read_csv('test.csv', sep='t')
                                        data.dropna(axis=1).to_csv('test_clean.csv')


                                        The important thing here is to add the axis=1 to tell Pandas to apply the dropna to columns instead of rows.






                                        share|improve this answer












                                        While trying different bash approaches I needed to remove all empty columns (including the header) reliably. To solve this I used Python with Pandas.



                                        import pandas as pd

                                        data = pd.read_csv('test.csv', sep='t')
                                        data.dropna(axis=1).to_csv('test_clean.csv')


                                        The important thing here is to add the axis=1 to tell Pandas to apply the dropna to columns instead of rows.







                                        share|improve this answer












                                        share|improve this answer



                                        share|improve this answer










                                        answered Aug 3 at 8:41









                                        Gawin

                                        1032




                                        1032























                                            0














                                            For a typical user the easiest way would be to import data in Excel from this CSV file and export it once again after removing the column.






                                            share|improve this answer


























                                              0














                                              For a typical user the easiest way would be to import data in Excel from this CSV file and export it once again after removing the column.






                                              share|improve this answer
























                                                0












                                                0








                                                0






                                                For a typical user the easiest way would be to import data in Excel from this CSV file and export it once again after removing the column.






                                                share|improve this answer












                                                For a typical user the easiest way would be to import data in Excel from this CSV file and export it once again after removing the column.







                                                share|improve this answer












                                                share|improve this answer



                                                share|improve this answer










                                                answered Nov 4 at 17:53









                                                pbies

                                                1,54711217




                                                1,54711217






























                                                    draft saved

                                                    draft discarded




















































                                                    Thanks for contributing an answer to Super User!


                                                    • Please be sure to answer the question. Provide details and share your research!

                                                    But avoid



                                                    • Asking for help, clarification, or responding to other answers.

                                                    • Making statements based on opinion; back them up with references or personal experience.


                                                    To learn more, see our tips on writing great answers.





                                                    Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                                                    Please pay close attention to the following guidance:


                                                    • Please be sure to answer the question. Provide details and share your research!

                                                    But avoid



                                                    • Asking for help, clarification, or responding to other answers.

                                                    • Making statements based on opinion; back them up with references or personal experience.


                                                    To learn more, see our tips on writing great answers.




                                                    draft saved


                                                    draft discarded














                                                    StackExchange.ready(
                                                    function () {
                                                    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f750651%2fdelete-empty-column-from-csv-file-with-bash-script%23new-answer', 'question_page');
                                                    }
                                                    );

                                                    Post as a guest















                                                    Required, but never shown





















































                                                    Required, but never shown














                                                    Required, but never shown












                                                    Required, but never shown







                                                    Required, but never shown

































                                                    Required, but never shown














                                                    Required, but never shown












                                                    Required, but never shown







                                                    Required, but never shown







                                                    Popular posts from this blog

                                                    Plaza Victoria

                                                    Puebla de Zaragoza

                                                    Musa