Removing header from CSV file

I had a 4 GB csv file from which I wanted to remove the header.  I tried using DOS for doing so as I was not able to open the file in any text editor or in Excel.  Within DOS, I tried the following

MORE +1 DATA.csv > output.csv

The above command would create a file output.csv but it would only contain 13000 rows and then just hang.  I tried creating a batch file to accomplish it but again the same problem arised.  Finally, I fired up my Cygwin terminal and used the following command

sed -i 1d DATA.csv

The UNIX command above will basically modify the file in-place and remove the first line as 1d stands for line number where the action will take place (1 in this case) and the action intended (d stands for delete here).  The problem with sed is that even though you only need to remove line 1, the command will parse through the full file.  A similar option to achieve the same in UNIX is :

awk ‘NR>1 {print}’

Advertisements

Got something to say?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s