none
Create ZIP of CSV files RRS feed

  • Question

  • I am working in .Net 4.7.2. We have List of objects say MyObject which is to be converted to csv single file. Currently using below code I create HUGE csv file ( 10GB and onwards).

    using (var writ = new StreamWriter(fileStream, Encoding.UTF8)) { using (var csvWrit = new CsvWriter(writ)) { //logic to read one object

    // write to csv

    csvWrit.NextRecord(); } } ZipFile.CreateFromDirectory(<sourcefileName>, <destFileName>);

    Now i need to create zip of these HUGE file. I found ZipFile.CreateFromDirectory in C#. After csv is created i call ZipFile.CreateFromDirectory to create zip file.

    My Question Should i continue to first create the csv and then zip OR Any other efficient way to do this?



    • Edited by Dotnetavalanche Friday, June 29, 2018 4:37 PM code format improved
    Friday, June 29, 2018 4:36 PM

Answers

  • The next fragment seems to work:

    using( var zf = ZipFile.Open( "myfile.zip", ZipArchiveMode.Create ) )
    {
       var ze = zf.CreateEntry( "myfile.csv" );
       using( var zs = ze.Open() )
       {
          using( var writ = new StreamWriter( zs, Encoding.UTF8 ) )
          {
             // write to 'writ'
             // . . .
          }
       }
    }


    Friday, June 29, 2018 8:29 PM

All replies

  • .zip files are just compressed files so if you want a .zip file you'd need the csv file first. However if you just need to compress data and you don't care about it being extractable to a csv then you can also consider using one of the compression stream classes instead. It would allow you to stream the data directly to a compressed file. DeflateStream is based upon zlib and is pretty fast. Note that these types of classes aren't designed to produce files you can then extract files from, they are designed to compress data so the resulting file would be smaller but you wouldn't just be able to open the file and access the underlying data.

    Michael Taylor http://www.michaeltaylorp3.net

    Friday, June 29, 2018 5:43 PM
    Moderator
  • The next fragment seems to work:

    using( var zf = ZipFile.Open( "myfile.zip", ZipArchiveMode.Create ) )
    {
       var ze = zf.CreateEntry( "myfile.csv" );
       using( var zs = ze.Open() )
       {
          using( var writ = new StreamWriter( zs, Encoding.UTF8 ) )
          {
             // write to 'writ'
             // . . .
          }
       }
    }


    Friday, June 29, 2018 8:29 PM
  • Hi  Dotnetavalanche,

    I Have a requirement to export data as csv, zip it and give it for download. So, considering the huge data around 100,000+ records, is it best to use the above code or do we need to choose a different path? please suggest.

    Thanks,

    Vamsi K


    Regards, Vamsi Kethineni

    Tuesday, November 5, 2019 5:49 AM