locked
Padding is invalid and cannot be removed while decrypt data in chunks and return the same gives RRS feed

  • Question

  • Hello,

    I am using  RijndaelManaged to encrypt and decrypt the data.

    Below is my requirement

    Step 1 : My source file size is 100 MB.

    Step 2 : I encrypt source file data in 10 MB chunk and saving it in on temp File. So once encryption is over, my temp file size would be 100 MB.

    Step 3 : I want to fetch 10-10 MB from temp-encrypted file and want to send 10-10 MB to my another class. i.e 

    • fetch 10 MB encrypted data
    • decrypt it
    • Send to SomeClass.
    • Again fetch 10 MB, decrypt and send to some method till i reach 0.

    Encryption is perfectly working but while decrypting it gives error "Padding is invalid and can not be removed". may be because i am trying to decrypt 10 mb and exiting from location. I am using same Key and IV to encrypt and decrypt data.

    below is my encryption code

    ==========================================================

    private void WriteDatainChunks(Stream stream, string tempFilePath)
            {   
                int chunkSize = 50 * 1024 * 1024;
                byte[] buffer = new byte[chunkSize];

                using (var rijndaelManaged = new RijndaelManaged())
                {
                    var encryptor = rijndaelManaged.CreateEncryptor(EncryptionKey, EncryptionIV);

                    using (var fileStream = File.Create(filePath))
                    {
                        using (var cryptoStream = new CryptoStream(fileStream, encryptor, CryptoStreamMode.Write))
                        {
                            int bytesRead;
                            while ((bytesRead = fsstream.Read(buffer, 0, chunkSize)) > 0)
                            {
                                cryptoStream.Write(buffer, 0, bytesRead);
                            }

                            if (!cryptoStream.HasFlushedFinalBlock)
                                cryptoStream.FlushFinalBlock();
                        }
                    }
                }
            }

    Below code that i am trying 

    ============================

    public byte[] GetDecryptedFileContent(string tempFilePath, long chunkSizeInBytes, long seekValue, long fileSize, string)
            {
                var sourceFile = new FileInfo(filePath);
                var buffer = new byte[chunkSizeInBytes];

                using (var fileStream = File.Open(sourceFile.FullName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
                {
                    fileStream.Seek(seekValue, SeekOrigin.Begin);
                    using (var binaryReader = new BinaryReader(fileStream))
                    {
                        binaryReader.Read(buffer, 0, buffer.Length);
                    }
                }

                using (MemoryStream memoryStream = new MemoryStream(buffer))
                {
                    using (MemoryStream dataOut = new MemoryStream())
                    {
                        using (RijndaelManaged rijAlg = new RijndaelManaged())
                        {
                            ICryptoTransform decryptor = rijAlg.CreateDecryptor(encryptionKey, encryptionIV);

                            using (CryptoStream cryptoStream = new CryptoStream(memoryStream, decryptor, CryptoStreamMode.Read))
                            {
                                using (BinaryReader decryptedData = new BinaryReader(cryptoStream))
                                {
                                    byte[] buffer1 = new byte[chunkSizeInBytes];
                                    int count;
                                    while ((count = decryptedData.Read(buffer1, 0, buffer1.Length)) != 0)
                                    {
                                        dataOut.Write(buffer1, 0, count);
                                    }
                                    return dataOut.ToArray();
                                }
                            }
                        }
                    }
                }
                return null;
            }

    ===================================

    If i decrypt entire file in chunks and return entire contents then it perfectly working fine. Below decryption code is working fine because i am decrypting entire file and entire contents i am returning.

                   

     private static byte[] ReadFileInChunks(string outputFile, int chunkSize, byte[] encryptionKey, byte[] encryptionIV)
            {
                chunkSize = 50 * 1024 * 1024;
                byte[] buffer = new byte[chunkSize];

                using (var fileReader = File.Open(outputFile, FileMode.Open, FileAccess.Read))
                {
                    if (fileReader.Length <= 0) 
                        return new byte[0];

                    using (var dataOut = new MemoryStream())
                    {
                        using (var rijndaelManaged = new RijndaelManaged())
                        {
                            var decryptor = rijndaelManaged.CreateDecryptor(encryptionKey, encryptionIV);
                            using (var cryptoStream = new CryptoStream(fileReader, decryptor, CryptoStreamMode.Read))
                            {
                                int bytesRead;
                                do
                                {
                                    bytesRead = cryptoStream.Read(buffer, 0, buffer.Length);
                                    dataOut.Write(buffer, 0, bytesRead);
                                }
                                while (bytesRead > 0);
                                return dataOut.ToArray();
                            }
                        }
                    }
                }
            }

    But my requirement is read encrypted data in chunks, decrypt it, send it. Again do the same process.

    Can someone please help me to resolve the issue? I tried by setting padding mode to zeros and non but didn't work.



    Friday, September 15, 2017 11:07 AM

All replies

  • Hi Dharmesh Fichadiya,

    Thank you for posting here.

    What is the key length and the block size in your code?

    The RijndaelManaged algorithm supports key lengths of 128, 192, or 256 bits; defaulting to 256 bits. This algorithm supports block sizes of 128, 192, or 256 bits; defaulting to 128 bits (Aes-compatible).

    Can you give it a try by explicitly setting the padding mode to be identical on both the encryption and decryption? For example:

    rijAlg.Padding = PaddingMode.NONE;

    You could refer to the following link for reference.

     

    https://stackoverflow.com/questions/8583112/padding-is-invalid-and-cannot-be-removed

    https://stackoverflow.com/questions/23406135/error-rijndaelmanaged-padding-is-invalid-and-cannot-be-removed

    The Rijndael class is the predecessor of the Aes algorithm. I suggest you to use the Aes algorithm instead of Rijndael. For more information, see the entry The Differences Between Rijndael and AES in the .NET Security blog.

    Best Regards,

    Wendy



    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.


    Tuesday, September 19, 2017 6:33 AM
  • Hello Wendy,

    Thanks for your response. I have already tried by setting padding mode to zero. Please refer below POC code it will clear all the things what i am trying to do. below are the steps/story board for same

    1) Encrypt source file and write contents to encrypted file in one go.

    2) Decrypt encrypted file in chunks.

    i.e. my file size is 10 MB and chunk size is 5 MB. So decrypt 5 MB and write it to output file. Again decrypt 5 MB and write it to output file. till encrypted file size becomes 0.

    3) I am using hard coded key generated from symmetricAlgo.GenerateKey of size byte[32] for encryption, decryption and for all chunks

    4) IV size is byte[16] for all chunks

    5) Key and IV is same for encryption and decryption.

    6) While encrypting, i am flushing final block.

                

    static void Main(string[] args)
            {
                string inputFile = @"C:\TestFile_txt10MB.log";
                string encryptFile = @"C:\\Encrypt.txt";
                string decryptFile = @"C:\\Decrypt.txt";
                string keyFile = @"C:\\Encryption_key.dat";

                if (!File.Exists(keyFile))
                {
                    //Hardcoded encryption key generated using symmetric.GenerateKey
                    File.WriteAllBytes(keyFile, Convert.FromBase64String("mqEUliuY2poyjwzcldcwvAXpkrFqshXct0hjChU7+K4="));
                }

                RijndaelManaged rijndaelManaged = new RijndaelManaged();             
                rijndaelManaged.GenerateIV();

                byte[] key = File.ReadAllBytes(keyFile);
                byte[] iv = rijndaelManaged.IV;

                EncryptFileInOneGo(inputFile, encryptFile, key, iv);

                var largeFile = new FileInfo(inputFile);
                long fileLength = largeFile.Length;
                long chunkSizeInBytes = 5 * 1024 * 1024;

                if (fileLength > chunkSizeInBytes)
                {
                    using (FileStream fileStream = new FileStream(decryptFile, FileMode.Create, System.IO.FileAccess.Write))
                    {
                        using (BinaryWriter writer = new BinaryWriter(fileStream))
                        {
                            long length = 0;
                            while (fileLength > 0)
                            {
                                var receivedBytes = DecryptFileInChunks(encryptFile, chunkSizeInBytes, length, key, iv);
                                writer.BaseStream.Seek(length, SeekOrigin.Begin);
                                writer.Write(receivedBytes);

                                length += receivedBytes.Length;
                                #region Append Memory Stream

                                #endregion
                                fileLength -= chunkSizeInBytes;
                                chunkSizeInBytes = (fileLength > chunkSizeInBytes) ? chunkSizeInBytes : fileLength;
                            }
                        }
                    }
                }
            }

    ======================================================
            private static void EncryptFileInOneGo(string inputFile, string outputFile, byte[] key, byte[] iv)
            {
                int chunkSize = 5 * 1024 * 1024;
                byte[] buffer = new byte[chunkSize];

                using (var rijndaelManaged = new RijndaelManaged())
                {
                    var encryptor = rijndaelManaged.CreateEncryptor(key, iv);

                    using (var fileStream = File.Create(outputFile))
                    {
                        using (var cryptoStream = new CryptoStream(fileStream, encryptor, CryptoStreamMode.Write))
                        {
                            using (var fsIn = new FileStream(inputFile, FileMode.Open))
                            {
                                int bytesRead;
                                while ((bytesRead = fsIn.Read(buffer, 0, chunkSize)) > 0)
                                {
                                    cryptoStream.Write(buffer, 0, bytesRead);
                                }
                            }

                            if (!cryptoStream.HasFlushedFinalBlock)
                                cryptoStream.FlushFinalBlock();
                        }
                    }
                }
            }

    =======================================================
         /// Decrypt encrypted file in chunks

    =======================================        

            public static byte[] DecryptFileInChunks(string encryptedFile, long chunkSizeInBytes, long seekValue, byte[] key, byte[] iv)
            {
                var sourceFile = new FileInfo(encryptedFile);
                var buffer = new byte[chunkSizeInBytes];
                int bytesRead = 0;

                using (var fileStream = File.Open(sourceFile.FullName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
                {
                    fileStream.Seek(seekValue, SeekOrigin.Begin);

                    using (MemoryStream dataOut = new MemoryStream())
                    {
                        using (RijndaelManaged rijndaelManaged = new RijndaelManaged())
                        {
                            rijndaelManaged.Padding = PaddingMode.None;
                            ICryptoTransform decryptor = rijndaelManaged.CreateDecryptor(key, iv);

                            using (CryptoStream cryptoStream = new CryptoStream(dataOut, decryptor, CryptoStreamMode.Write))
                            {

                                bytesRead = fileStream.Read(buffer, 0, buffer.Length);
                                cryptoStream.Write(buffer, 0, bytesRead);
                                return dataOut.ToArray();
                            }
                        }
                    }
                }
                return null;
            }

    When i execute above code,

    1) Encryption works fine

    2) In decryption it gives error "Padding is invalid"

    3) In decryption, if i give padding mode none then for LAST CHUNK, i am getting an error "Length of the data to decrypt is invalid". So i am little bit stuck. 

    Went through lot of articles but all are encrypting and decrypting in one go. I want to decrypt in chunks only because i have to copy large file i.e. 14GB. 

    Can you please go through the code i am posting and share your valuable feedback?






    Tuesday, September 19, 2017 10:11 AM
  • Hi Dharmesh Fichadiya,

    Most plain text messages do not consist of a number of bytes that completely fill blocks. Some encryption standards specify a particular padding scheme. Have you tried another padding mode? Please refer to the following link to try another four types of padding mode.

    https://msdn.microsoft.com/en-us/library/system.security.cryptography.paddingmode.aspx

    Maybe you could try the decryption without padding mode.

    Best Regards,

    Wendy


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Sunday, September 24, 2017 3:38 PM