none
ZipFile CreateFromDirectory creating corrupted archive? RRS feed

  • Question

  • Hello,

    Our service is processing huge amount of data and needs to do quite verbose logging - so we produce few dozens GB of log data per day. After day end we archive our previous day logs by using this function:

    private static void ArchiveDirectory(string target_dir)
    {
    	try
    	{
    		if (Directory.Exists(target_dir) && !File.Exists(target_dir + ".zip"))
    		{
    			System.IO.Compression.ZipFile.CreateFromDirectory(target_dir, target_dir + ".zip");
    			DeleteDirectory(target_dir);
    		}
    	}
    	catch (Exception)
    	{ /* This is intentional as there is no place to log during log archival */ } 
    }      
           

     It happened few times that the log files were archived into an archive, original folder was deleted, but the archive was corrupted (some files couldn't be opened (Windows giving 0x80004005 - Unspecified Error) and archive couldn't be completely unpacked, archive validation tools (e.g. winrar) reported corruption).

    Is it possible that ZipFile.CreateFromDirectory can create corrupted archives without throwing an exception? How can one programaticaly validate the archive before proceeding with deletion?

    Thanks
    Jan

    Tuesday, June 25, 2013 10:24 AM

All replies

  • Hi Jan,

    >>Is it possible that ZipFile.CreateFromDirectory can create corrupted archives without throwing an exception

    You can use a try-catch block to surrounding the code, and handle it, then, it will not throw out and stop the application.

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Wednesday, June 26, 2013 2:17 PM
    Moderator
  • So the problem turned out to be with the target .zip file size. It was greater than 4GB - which cannot be correctly opened by built-in windows zip support, or many compression tools.
    System.IO.Compression.ZipFile.ExtractToDirectory can still extract the zip without the problems.

    Thanks
    Jan

    • Marked as answer by Jan Krivanek Wednesday, June 26, 2013 2:41 PM
    • Unmarked as answer by Jan Krivanek Thursday, July 18, 2013 4:08 PM
    Wednesday, June 26, 2013 2:40 PM
  • The only remaining question now is how to open those zip archives >4GB. I've tried many tools supporting zip64 format with no success.

    There is no comment about this situation in MSDN documentation, but it seems that the only possibility for created .zip files bigger than 4GB is to use System.IO.Compression functions (= writing proprietary custom tool).

    Or is there any recommended way how to open .zip iles bigger then 4GB created by System.IO.Compression.ZipFile.CreateFromDirectory function? I gues this is something that is worth mentioning in the documentation.

    Thanks
    Jan

    Thursday, June 27, 2013 11:02 AM
  • We hit this again several times and we couldn't completely unpack the archive even with ExtractToDirectory function.
    It turns out that we can reliably and repeatedly replicate this:

    If we pack the logs directory with size over ~45GB by calling System.IO.Compression.ZipFile.CreateFromDirectory (operation finishes without any error), then unpacking by calling the reverse function System.IO.Compression.ZipFile.ExtractToDirectory - won't unpack the entire original data and will fail with exception:

    System.IO.InvalidDataException: A local file header is corrupt.
       at System.IO.Compression.ZipArchiveEntry.ThrowIfNotOpenable(Boolean needToUncompress, Boolean needToLoadIntoMemory)
       at System.IO.Compression.ZipArchiveEntry.OpenInReadMode(Boolean checkOpenable)
       at System.IO.Compression.ZipArchiveEntry.Open()
       at System.IO.Compression.ZipFileExtensions.ExtractToFile(ZipArchiveEntry source, String destinationFileName, Boolean overwrite)
       at System.IO.Compression.ZipFileExtensions.ExtractToDirectory(ZipArchive source, String destinationDirectoryName)
       at System.IO.Compression.ZipFile.ExtractToDirectory(String sourceArchiveFileName, String destinationDirectoryName, Encoding entryNameEncoding)
       at System.IO.Compression.ZipFile.ExtractToDirectory(String sourceArchiveFileName, String destinationDirectoryName)

    This seems to be a bug to me and it causes that we are losing access to some of our important log files. Can anybody - preferably owner of ZipFile class in CLR team - look into this?

    Thanks
    Jan

    Thursday, July 18, 2013 4:16 PM