locked
Convert.FromBase64String() throws System.OutOfMemoryException RRS feed

  • Question

  • Hi everyBody,

     

    I am trying to work with a large amount of data that is encoded in Base64. I have a method that I use to decode an array of bytes into another array, that is not encrypted. See my method below :

     

     

     public static byte[] Base64Decode(byte[] encrypted64)
        {
          string str = System.Text.Encoding.Default.GetString(encrypted64);
    
          if(str.Trim().Length > 0)
            return Convert.FromBase64String(str);
          else
            return new byte[] { };
        }

     

    Actually, I use it to decode PDF files that are embedded into emails, in MIME format. Most of the time, it works fine, but when the amount of data is too large, I get a System.OutOfMemoryException ! It appends when the PDF files are larger thant 20 MB.

    I have no control on the Convert.FromBase64String, because it's a Framework method. I would like to find an alternative that takes less memory.

    Can you help me please ?

     

    Best regards.

     

    Matteo


    Matteo, .NET Developer
    Monday, April 11, 2011 7:26 AM

Answers

  • Thanks Jesse !

     

    Based on your article, I decided to modify the encoder avaliable on The Code Project. To avoid having System.OutOfMemoryException thrown, I use file streams instead of the RAM.

     

    I published the code so that the developer can use it if he wants. See the code on The Code Project artcile page, under "Comments and discussions" sections.

     

    http://www.codeproject.com/KB/cs/base64encdec.aspx?msg=3870543#xx3870543xx

     

    I hope this will help people having the same issue.

     

    Best regards.


    Matteo, .NET Developer
    • Marked as answer by joujoukinder Wednesday, April 27, 2011 2:21 PM
    Wednesday, April 27, 2011 2:21 PM
  • Hi Matteo,

     

    You may try to use FromBase64Transform method.

    The FromBase64String method is designed to process a single string that contains all the data to be decoded. To decode base-64 character data from a stream, use the System.Security.Cryptography.FromBase64Transform class.

     

    Have a nice day!


    Paul Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Proposed as answer by Albin Abel Wednesday, April 13, 2011 12:02 PM
    • Marked as answer by Paul Zhou Tuesday, April 26, 2011 3:07 AM
    Wednesday, April 13, 2011 8:11 AM
  • If you're willing to modify the input you can do

    private int RemoveInvalidChars(char[] input)
    {
    int index = 0;
    for (int i = 0; i < input.Length; i++)
    {
      if (!IsInvalidChar(input[i]))
      {
        input[index] = input[i];
        index++;
      }
    }
    for (int j = index; j < input.Length; j++)
    {
      input[j] = Char.MinValue;
    }
    return index;
    }
    

    • Marked as answer by Paul Zhou Tuesday, April 26, 2011 3:07 AM
    Tuesday, April 19, 2011 5:03 PM
  • Check out this article, which explains how to implement your own Encoder class which you can plug into the standard Stream Reader/Writer classes of the BCL:

    http://blogs.microsoft.co.il/blogs/kim/archive/2007/10/09/base64-encode-large-files-very-large-files.aspx

    It still have a few things that aren't working fully, but it is a very nice place to start.

    • Marked as answer by Paul Zhou Tuesday, April 26, 2011 3:07 AM
    Tuesday, April 19, 2011 5:13 PM

All replies

  • Base64 is based on a standard so you can find many Base64 Encoder and Decoders online. One such is here: http://www.codeproject.com/KB/cs/base64encdec.aspx

    The good thing is you get the source code, so you tweak it according to your needs and debug if it fails.


    -- Abhimanyu
    Monday, April 11, 2011 11:57 AM
  • Thanks Abhimanyu,

     

    Actually, I tried, but I don't get the same result using that classe and theone. It's because I am decoding a Base64 string that comes from a MIME email document and it contains \r chars. "Convert.FromBase64String(...)" gives me more bytes than the Base64Encoder of Code Project. I know that the "Convert.FromBase64String(...)" gives me the right bytes, because after saving on the hard disk I am able to open the file. With the Code Project class, the file is corrupted.

    The "Convert.FromBase64String(...)" gives me more bytes, I guess it encodes the \r char. But I don't know how to reproduce this.

    Below, the base64 string, and then both the byte arrays, first the one of "Convert.FromBase64String(...)" and the the one of the Code Project class :

     

    The Base64 string :
    =================
    
    JVBERi0xLjQKJcfsj6IKNSAwIG9iago8PC9MZW5ndGggNiAwIFIvRmlsdGVyIC9GbGF0ZURlY29k
    ZT4+CnN0cmVhbQp4nE2LsQoCQQxE+3xFyt3iYrK7d9m0ggh2SjqxOtRqi0P/H/cOFGcYePCYBZkk
    Ia/9wtyA8dj3hAUq5TWb+Oe54d5hd1HsL38Ak5kV2ZygMlXUiamgN7iGdxwSyTSONdxjJi3hFQeh
    lMz0J3PAePMTHBzOvR9KJB8fZW5kc3RyZWFtCmVuZG9iago2IDAgb2JqCjEyMQplbmRvYmoKNCAw
    IG9iago8PC9UeXBlL1BhZ2UvTWVkaWFCb3ggWzAgMCA1OTUgODQyXQovUm90YXRlIDAvUGFyZW50
    IDMgMCBSCi9SZXNvdXJjZXM8PC9Qcm9jU2V0Wy9QREYgL1RleHRdCi9Gb250IDggMCBSCj4+Ci9D
    b250ZW50cyA1IDAgUgo+PgplbmRvYmoKMyAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWwo0
    IDAgUgpdIC9Db3VudCAxCj4+CmVuZG9iagoxIDAgb2JqCjw8L1R5cGUgL0NhdGFsb2cgL1BhZ2Vz
    IDMgMCBSCi9NZXRhZGF0YSA5IDAgUgo+PgplbmRvYmoKOCAwIG9iago8PC9SNwo3IDAgUj4+CmVu
    ZG9iago3IDAgb2JqCjw8L0Jhc2VGb250L1RpbWVzLVJvbWFuL1R5cGUvRm9udAovU3VidHlwZS9U
    eXBlMT4+CmVuZG9iago5IDAgb2JqCjw8L1R5cGUvTWV0YWRhdGEKL1N1YnR5cGUvWE1ML0xlbmd0
    aCAxNzE3Pj5zdHJlYW0KPD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6
    TlRjemtjOWQnPz4KPD9hZG9iZS14YXAtZmlsdGVycyBlc2M9IkNSTEYiPz4KPHg6eG1wbWV0YSB4
    bWxuczp4PSdhZG9iZTpuczptZXRhLycgeDp4bXB0az0nWE1QIHRvb2xraXQgMi45LjEtMTMsIGZy
    YW1ld29yayAxLjYnPgo8cmRmOlJERiB4bWxuczpyZGY9J2h0dHA6Ly93d3cudzMub3JnLzE5OTkv
    MDIvMjItcmRmLXN5bnRheC1ucyMnIHhtbG5zOmlYPSdodHRwOi8vbnMuYWRvYmUuY29tL2lYLzEu
    MC8nPgo8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0nZmJmZDc5N2MtNjY5OC0xMWUwLTAwMDAt
    MjU5ZDI4ODI0NjUyJyB4bWxuczpwZGY9J2h0dHA6Ly9ucy5hZG9iZS5jb20vcGRmLzEuMy8nPjxw
    ZGY6UHJvZHVjZXI+R1BMIEdob3N0c2NyaXB0IDguNzA8L3BkZjpQcm9kdWNlcj4KPHBkZjpLZXl3
    b3Jkcz4oKTwvcGRmOktleXdvcmRzPgo8L3JkZjpEZXNjcmlwdGlvbj4KPHJkZjpEZXNjcmlwdGlv
    biByZGY6YWJvdXQ9J2ZiZmQ3OTdjLTY2OTgtMTFlMC0wMDAwLTI1OWQyODgyNDY1MicgeG1sbnM6
    eG1wPSdodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvJz48eG1wOk1vZGlmeURhdGU+MjAxMS0w
    NC0xMVQxNToxMzoyMiswMjowMDwveG1wOk1vZGlmeURhdGU+Cjx4bXA6Q3JlYXRlRGF0ZT4yMDEx
    LTA0LTExVDE1OjEzOjIyKzAyOjAwPC94bXA6Q3JlYXRlRGF0ZT4KPHhtcDpDcmVhdG9yVG9vbD5c
    Mzc2XDM3N1wwMDBQXDAwMERcMDAwRlwwMDBDXDAwMHJcMDAwZVwwMDBhXDAwMHRcMDAwb1wwMDBy
    XDAwMCBcMDAwVlwwMDBlXDAwMHJcMDAwc1wwMDBpXDAwMG9cMDAwblwwMDAgXDAwMDFcMDAwLlww
    MDAwXDAwMC5cMDAwMjwveG1wOkNyZWF0b3JUb29sPjwvcmRmOkRlc2NyaXB0aW9uPgo8cmRmOkRl
    c2NyaXB0aW9uIHJkZjphYm91dD0nZmJmZDc5N2MtNjY5OC0xMWUwLTAwMDAtMjU5ZDI4ODI0NjUy
    JyB4bWxuczp4YXBNTT0naHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLycgeGFwTU06RG9j
    dW1lbnRJRD0nZmJmZDc5N2MtNjY5OC0xMWUwLTAwMDAtMjU5ZDI4ODI0NjUyJy8+CjxyZGY6RGVz
    Y3JpcHRpb24gcmRmOmFib3V0PSdmYmZkNzk3Yy02Njk4LTExZTAtMDAwMC0yNTlkMjg4MjQ2NTIn
    IHhtbG5zOmRjPSdodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLycgZGM6Zm9ybWF0PSdh
    cHBsaWNhdGlvbi9wZGYnPjxkYzp0aXRsZT48cmRmOkFsdD48cmRmOmxpIHhtbDpsYW5nPSd4LWRl
    ZmF1bHQnPlwzNzZcMzc3XDAwMERcMDAwb1wwMDBjXDAwMHVcMDAwbVwwMDBlXDAwMG5cMDAwdFww
    MDAxPC9yZGY6bGk+PC9yZGY6QWx0PjwvZGM6dGl0bGU+PGRjOmNyZWF0b3I+PHJkZjpTZXE+PHJk
    ZjpsaT5cMzc2XDM3N1wwMDBzXDAwMGVcMDAwbFwwMDB2XDAwMG1cMDAwYVwwMDB0XDAwMHQ8L3Jk
    ZjpsaT48L3JkZjpTZXE+PC9kYzpjcmVhdG9yPjxkYzpkZXNjcmlwdGlvbj48cmRmOlNlcT48cmRm
    OmxpPigpPC9yZGY6bGk+PC9yZGY6U2VxPjwvZGM6ZGVzY3JpcHRpb24+PC9yZGY6RGVzY3JpcHRp
    b24+CjwvcmRmOlJERj4KPC94OnhtcG1ldGE+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
    ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAg
    ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
    ICAKPD94cGFja2V0IGVuZD0ndyc/PgplbmRzdHJlYW0KZW5kb2JqCjIgMCBvYmoKPDwvUHJvZHVj
    ZXIoR1BMIEdob3N0c2NyaXB0IDguNzApCi9DcmVhdGlvbkRhdGUoRDoyMDExMDQxMTE1MTMyMisw
    MicwMCcpCi9Nb2REYXRlKEQ6MjAxMTA0MTExNTEzMjIrMDInMDAnKQovVGl0bGUoXDM3NlwzNzdc
    MDAwRFwwMDBvXDAwMGNcMDAwdVwwMDBtXDAwMGVcMDAwblwwMDB0XDAwMDEpCi9DcmVhdG9yKFwz
    NzZcMzc3XDAwMFBcMDAwRFwwMDBGXDAwMENcMDAwclwwMDBlXDAwMGFcMDAwdFwwMDBvXDAwMHJc
    MDAwIFwwMDBWXDAwMGVcMDAwclwwMDBzXDAwMGlcMDAwb1wwMDBuXDAwMCBcMDAwMVwwMDAuXDAw
    MDBcMDAwLlwwMDAyKQovQXV0aG9yKFwzNzZcMzc3XDAwMHNcMDAwZVwwMDBsXDAwMHZcMDAwbVww
    MDBhXDAwMHRcMDAwdCkKL0tleXdvcmRzKCkKL1N1YmplY3QoKT4+ZW5kb2JqCnhyZWYKMCAxMAow
    MDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDA0MjUgMDAwMDAgbiAKMDAwMDAwMjM3NyAwMDAwMCBu
    IAowMDAwMDAwMzY2IDAwMDAwIG4gCjAwMDAwMDAyMjUgMDAwMDAgbiAKMDAwMDAwMDAxNSAwMDAw
    MCBuIAowMDAwMDAwMjA2IDAwMDAwIG4gCjAwMDAwMDA1MTggMDAwMDAgbiAKMDAwMDAwMDQ4OSAw
    MDAwMCBuIAowMDAwMDAwNTg0IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgMTAgL1Jvb3QgMSAw
    IFIgL0luZm8gMiAwIFIKL0lEIFs8ODU0RDRCM0EyODk2MEIxNDQyNjBDMEY4REQ5MDY0Mzc+PDg1
    NEQ0QjNBMjg5NjBCMTQ0MjYwQzBGOEREOTA2NDM3Pl0KPj4Kc3RhcnR4cmVmCjI3ODIKJSVFT0YK
    

    The "Convert.FromBase64String(...)" result :
    =====================
    
    37,80,68,70,45,49,46,52,10,37,199,236,143,162,10,53,32,48,32,111,98,106,10,60,60,47,76,101,110,103,116,104,32,54,32,48,32,82,47,70,105,108,116,101,114,32,47,70,108,97,116,101,68,101,99,111,100,101,62,62,10,115,116,114,101,97,109,10,120,156,77,139,177,10,2,65,12,68,251,124,69,202,221,226,98,178,187,119,217,180,130,8,118,74,58,177,58,212,106,139,67,255,31,247,14,20,103,24,120,240,152,5,153,36,33,175,253,194,220,128,241,216,247,132,5,42,229,53,155,248,231,185,225,222,97,119,81,236,47,127,0,147,153,21,217,156,160,50,85,212,137,169,160,55,184,134,119,28,18,201,52,142,53,220,99,38,45,225,21,7,161,148,204,244,39,115,192,120,243,19,28,28,206,189,31,74,36,31,31,101,110,100,115,116,114,101,97,109,10,101,110,100,111,98,106,10,54,32,48,32,111,98,106,10,49,50,49,10,101,110,100,111,98,106,10,52,32,48,32,111,98,106,10,60,60,47,84,121,112,101,47,80,97,103,101,47,77,101,100,105,97,66,111,120,32,91,48,32,48,32,53,57,53,32,56,52,50,93,10,47,82,111,116,97,116,101,32,48,47,80,97,114,101,110,116,32,51,32,48,32,82,10,47,82,101,115,111,117,114,99,101,115,60,60,47,80,114,111,99,83,101,116,91,47,80,68,70,32,47,84,101,120,116,93,10,47,70,111,110,116,32,56,32,48,32,82,10,62,62,10,47,67,111,110,116,101,110,116,115,32,53,32,48,32,82,10,62,62,10,101,110,100,111,98,106,10,51,32,48,32,111,98,106,10,60,60,32,47,84,121,112,101,32,47,80,97,103,101,115,32,47,75,105,100,115,32,91,10,52,32,48,32,82,10,93,32,47,67,111,117,110,116,32,49,10,62,62,10,101,110,100,111,98,106,10,49,32,48,32,111,98,106,10,60,60,47,84,121,112,101,32,47,67,97,116,97,108,111,103,32,47,80,97,103,101,115,32,51,32,48,32,82,10,47,77,101,116,97,100,97,116,97,32,57,32,48,32,82,10,62,62,10,101,110,100,111,98,106,10,56,32,48,32,111,98,106,10,60,60,47,82,55,10,55,32,48,32,82,62,62,10,101,110,100,111,98,106,10,55,32,48,32,111,98,106,10,60,60,47,66,97,115,101,70,111,110,116,47,84,105,109,101,115,45,82,111,109,97,110,47,84,121,112,101,47,70,111,110,116,10,47,83,117,98,116,121,112,101,47,84,121,112,101,49,62,62,10,101,110,100,111,98,106,10,57,32,48,32,111,98,106,10,60,60,47,84,121,112,101,47,77,101,116,97,100,97,116,97,10,47,83,117,98,116,121,112,101,47,88,77,76,47,76,101,110,103,116,104,32,49,55,49,55,62,62,115,116,114,101,97,109,10,60,63,120,112,97,99,107,101,116,32,98,101,103,105,110,61,39,239,187,191,39,32,105,100,61,39,87,53,77,48,77,112,67,101,104,105,72,122,114,101,83,122,78,84,99,122,107,99,57,100,39,63,62,10,60,63,97,100,111,98,101,45,120,97,112,45,102,105,108,116,101,114,115,32,101,115,99,61,34,67,82,76,70,34,63,62,10,60,120,58,120,109,112,109,101,116,97,32,120,109,108,110,115,58,120,61,39,97,100,111,98,101,58,110,115,58,109,101,116,97,47,39,32,120,58,120,109,112,116,107,61,39,88,77,80,32,116,111,111,108,107,105,116,32,50,46,57,46,49,45,49,51,44,32,102,114,97,109,101,119,111,114,107,32,49,46,54,39,62,10,60,114,100,102,58,82,68,70,32,120,109,108,110,115,58,114,100,102,61,39,104,116,116,112,58,47,47,119,119,119,46,119,51,46,111,114,103,47,49,57,57,57,47,48,50,47,50,50,45,114,100,102,45,115,121,110,116,97,120,45,110,115,35,39,32,120,109,108,110,115,58,105,88,61,39,104,116,116,112,58,47,47,110,115,46,97,100,111,98,101,46,99,111,109,47,105,88,47,49,46,48,47,39,62,10,60,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,32,114,100,102,58,97,98,111,117,116,61,39,102,98,102,100,55,57,55,99,45,54,54,57,56,45,49,49,101,48,45,48,48,48,48,45,50,53,57,100,50,56,56,50,52,54,53,50,39,32,120,109,108,110,115,58,112,100,102,61,39,104,116,116,112,58,47,47,110,115,46,97,100,111,98,101,46,99,111,109,47,112,100,102,47,49,46,51,47,39,62,60,112,100,102,58,80,114,111,100,117,99,101,114,62,71,80,76,32,71,104,111,115,116,115,99,114,105,112,116,32,56,46,55,48,60,47,112,100,102,58,80,114,111,100,117,99,101,114,62,10,60,112,100,102,58,75,101,121,119,111,114,100,115,62,40,41,60,47,112,100,102,58,75,101,121,119,111,114,100,115,62,10,60,47,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,62,10,60,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,32,114,100,102,58,97,98,111,117,116,61,39,102,98,102,100,55,57,55,99,45,54,54,57,56,45,49,49,101,48,45,48,48,48,48,45,50,53,57,100,50,56,56,50,52,54,53,50,39,32,120,109,108,110,115,58,120,109,112,61,39,104,116,116,112,58,47,47,110,115,46,97,100,111,98,101,46,99,111,109,47,120,97,112,47,49,46,48,47,39,62,60,120,109,112,58,77,111,100,105,102,121,68,97,116,101,62,50,48,49,49,45,48,52,45,49,49,84,49,53,58,49,51,58,50,50,43,48,50,58,48,48,60,47,120,109,112,58,77,111,100,105,102,121,68,97,116,101,62,10,60,120,109,112,58,67,114,101,97,116,101,68,97,116,101,62,50,48,49,49,45,48,52,45,49,49,84,49,53,58,49,51,58,50,50,43,48,50,58,48,48,60,47,120,109,112,58,67,114,101,97,116,101,68,97,116,101,62,10,60,120,109,112,58,67,114,101,97,116,111,114,84,111,111,108,62,92,51,55,54,92,51,55,55,92,48,48,48,80,92,48,48,48,68,92,48,48,48,70,92,48,48,48,67,92,48,48,48,114,92,48,48,48,101,92,48,48,48,97,92,48,48,48,116,92,48,48,48,111,92,48,48,48,114,92,48,48,48,32,92,48,48,48,86,92,48,48,48,101,92,48,48,48,114,92,48,48,48,115,92,48,48,48,105,92,48,48,48,111,92,48,48,48,110,92,48,48,48,32,92,48,48,48,49,92,48,48,48,46,92,48,48,48,48,92,48,48,48,46,92,48,48,48,50,60,47,120,109,112,58,67,114,101,97,116,111,114,84,111,111,108,62,60,47,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,62,10,60,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,32,114,100,102,58,97,98,111,117,116,61,39,102,98,102,100,55,57,55,99,45,54,54,57,56,45,49,49,101,48,45,48,48,48,48,45,50,53,57,100,50,56,56,50,52,54,53,50,39,32,120,109,108,110,115,58,120,97,112,77,77,61,39,104,116,116,112,58,47,47,110,115,46,97,100,111,98,101,46,99,111,109,47,120,97,112,47,49,46,48,47,109,109,47,39,32,120,97,112,77,77,58,68,111,99,117,109,101,110,116,73,68,61,39,102,98,102,100,55,57,55,99,45,54,54,57,56,45,49,49,101,48,45,48,48,48,48,45,50,53,57,100,50,56,56,50,52,54,53,50,39,47,62,10,60,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,32,114,100,102,58,97,98,111,117,116,61,39,102,98,102,100,55,57,55,99,45,54,54,57,56,45,49,49,101,48,45,48,48,48,48,45,50,53,57,100,50,56,56,50,52,54,53,50,39,32,120,109,108,110,115,58,100,99,61,39,104,116,116,112,58,47,47,112,117,114,108,46,111,114,103,47,100,99,47,101,108,101,109,101,110,116,115,47,49,46,49,47,39,32,100,99,58,102,111,114,109,97,116,61,39,97,112,112,108,105,99,97,116,105,111,110,47,112,100,102,39,62,60,100,99,58,116,105,116,108,101,62,60,114,100,102,58,65,108,116,62,60,114,100,102,58,108,105,32,120,109,108,58,108,97,110,103,61,39,120,45,100,101,102,97,117,108,116,39,62,92,51,55,54,92,51,55,55,92,48,48,48,68,92,48,48,48,111,92,48,48,48,99,92,48,48,48,117,92,48,48,48,109,92,48,48,48,101,92,48,48,48,110,92,48,48,48,116,92,48,48,48,49,60,47,114,100,102,58,108,105,62,60,47,114,100,102,58,65,108,116,62,60,47,100,99,58,116,105,116,108,101,62,60,100,99,58,99,114,101,97,116,111,114,62,60,114,100,102,58,83,101,113,62,60,114,100,102,58,108,105,62,92,51,55,54,92,51,55,55,92,48,48,48,115,92,48,48,48,101,92,48,48,48,108,92,48,48,48,118,92,48,48,48,109,92,48,48,48,97,92,48,48,48,116,92,48,48,48,116,60,47,114,100,102,58,108,105,62,60,47,114,100,102,58,83,101,113,62,60,47,100,99,58,99,114,101,97,116,111,114,62,60,100,99,58,100,101,115,99,114,105,112,116,105,111,110,62,60,114,100,102,58,83,101,113,62,60,114,100,102,58,108,105,62,40,41,60,47,114,100,102,58,108,105,62,60,47,114,100,102,58,83,101,113,62,60,47,100,99,58,100,101,115,99,114,105,112,116,105,111,110,62,60,47,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,62,10,60,47,114,100,102,58,82,68,70,62,10,60,47,120,58,120,109,112,109,101,116,97,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,10,60,63,120,112,97,99,107,101,116,32,101,110,100,61,39,119,39,63,62,10,101,110,100,115,116,114,101,97,109,10,101,110,100,111,98,106,10,50,32,48,32,111,98,106,10,60,60,47,80,114,111,100,117,99,101,114,40,71,80,76,32,71,104,111,115,116,115,99,114,105,112,116,32,56,46,55,48,41,10,47,67,114,101,97,116,105,111,110,68,97,116,101,40,68,58,50,48,49,49,48,52,49,49,49,53,49,51,50,50,43,48,50,39,48,48,39,41,10,47,77,111,100,68,97,116,101,40,68,58,50,48,49,49,48,52,49,49,49,53,49,51,50,50,43,48,50,39,48,48,39,41,10,47,84,105,116,108,101,40,92,51,55,54,92,51,55,55,92,48,48,48,68,92,48,48,48,111,92,48,48,48,99,92,48,48,48,117,92,48,48,48,109,92,48,48,48,101,92,48,48,48,110,92,48,48,48,116,92,48,48,48,49,41,10,47,67,114,101,97,116,111,114,40,92,51,55,54,92,51,55,55,92,48,48,48,80,92,48,48,48,68,92,48,48,48,70,92,48,48,48,67,92,48,48,48,114,92,48,48,48,101,92,48,48,48,97,92,48,48,48,116,92,48,48,48,111,92,48,48,48,114,92,48,48,48,32,92,48,48,48,86,92,48,48,48,101,92,48,48,48,114,92,48,48,48,115,92,48,48,48,105,92,48,48,48,111,92,48,48,48,110,92,48,48,48,32,92,48,48,48,49,92,48,48,48,46,92,48,48,48,48,92,48,48,48,46,92,48,48,48,50,41,10,47,65,117,116,104,111,114,40,92,51,55,54,92,51,55,55,92,48,48,48,115,92,48,48,48,101,92,48,48,48,108,92,48,48,48,118,92,48,48,48,109,92,48,48,48,97,92,48,48,48,116,92,48,48,48,116,41,10,47,75,101,121,119,111,114,100,115,40,41,10,47,83,117,98,106,101,99,116,40,41,62,62,101,110,100,111,98,106,10,120,114,101,102,10,48,32,49,48,10,48,48,48,48,48,48,48,48,48,48,32,54,53,53,51,53,32,102,32,10,48,48,48,48,48,48,48,52,50,53,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,50,51,55,55,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,51,54,54,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,50,50,53,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,48,49,53,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,50,48,54,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,53,49,56,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,52,56,57,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,53,56,52,32,48,48,48,48,48,32,110,32,10,116,114,97,105,108,101,114,10,60,60,32,47,83,105,122,101,32,49,48,32,47,82,111,111,116,32,49,32,48,32,82,32,47,73,110,102,111,32,50,32,48,32,82,10,47,73,68,32,91,60,56,53,52,68,52,66,51,65,50,56,57,54,48,66,49,52,52,50,54,48,67,48,70,56,68,68,57,48,54,52,51,55,62,60,56,53,52,68,52,66,51,65,50,56,57,54,48,66,49,52,52,50,54,48,67,48,70,56,68,68,57,48,54,52,51,55,62,93,10,62,62,10,115,116,97,114,116,120,114,101,102,10,50,55,56,50,10,37,37,69,79,70,10

    The Code Project Class result :
    =====================

    37,80,68,70,45,49,46,52,10,37,199,236,143,162,10,53,32,48,32,111,98,106,10,60,60,47,76,101,110,103,116,104,32,54,32,48,32,82,47,70,105,108,116,101,114,32,47,70,108,97,116,101,68,101,99,111,100,0,6,83,227,224,167,55,71,38,86,22,208,167,137,196,216,187,16,160,36,16,196,79,183,196,92,173,222,38,43,43,183,125,155,72,32,135,100,163,171,19,173,70,168,180,63,241,255,112,225,70,113,135,143,9,128,89,146,64,0,33,175,253,194,220,128,241,216,247,132,5,42,229,53,155,248,231,185,225,222,97,119,81,236,47,127,0,147,153,21,217,156,160,50,85,212,137,169,160,55,184,134,119,28,18,201,52,142,53,220,99,38,45,225,21,7,161,0,9,76,207,66,119,60,7,143,49,49,193,204,235,209,244,162,65,241,246,86,230,71,55,71,38,86,22,208,166,86,230,70,246,38,160,163,98,3,2,6,246,38,160,163,19,35,16,166,86,230,70,246,38,160,163,66,3,0,0,32,111,98,106,10,60,60,47,84,121,112,101,47,80,97,103,101,47,77,101,100,105,97,66,111,120,32,91,48,32,48,32,53,57,53,32,56,52,50,93,10,47,82,111,116,97,116,101,32,48,47,80,97,114,101,110,116,0,2,3,50,3,2,5,32,162,245,38,87,54,247,87,38,54,87,51,195,194,245,7,38,246,53,54,87,69,178,245,4,68,98,2,245,70,87,135,69,208,162,244,102,246,231,66,3,130,3,2,5,32,163,227,224,162,244,48,0,111,110,116,101,110,116,115,32,53,32,48,32,82,10,62,62,10,101,110,100,111,98,106,10,51,32,48,32,111,98,106,10,60,60,32,47,84,121,112,101,32,47,80,97,103,101,115,32,47,75,105,100,115,32,91,10,52,0,2,3,2,5,32,165,210,2,244,54,247,86,231,66,3,16,163,227,224,166,86,230,70,246,38,160,163,18,3,2,6,246,38,160,163,195,194,245,71,151,6,82,2,244,54,23,70,22,198,246,114,2,245,6,22,118,87,48,0,32,51,32,48,32,82,10,47,77,101,116,97,100,97,116,97,32,57,32,48,32,82,10,62,62,10,101,110,100,111,98,106,10,56,32,48,32,111,98,106,10,60,60,47,82,55,10,55,32,48,32,82,62,62,10,101,110,0,6,70,246,38,160,163,114,3,2,6,246,38,160,163,195,194,244,38,23,54,84,102,246,231,66,245,70,150,214,87,50,213,38,246,214,22,226,245,71,151,6,82,244,102,246,231,64,162,245,55,86,39,71,151,6,82,245,64,0,121,112,101,49,62,62,10,101,110,100,111,98,106,10,57,32,48,32,111,98,106,10,60,60,47,84,121,112,101,47,77,101,116,97,100,97,116,97,10,47,83,117,98,116,121,112,101,47,88,77,76,47,76,101,110,103,116,0,6,130,3,19,115,19,115,227,231,55,71,38,86,22,208,163,195,247,135,6,22,54,182,87,66,6,38,86,118,150,227,210,126,251,187,242,114,6,150,67,210,117,115,84,211,4,215,4,54,86,134,148,135,167,38,85,55,160,0,78,84,99,122,107,99,57,100,39,63,62,10,60,63,97,100,111,98,101,45,120,97,112,45,102,105,108,116,101,114,115,32,101,115,99,61,34,67,82,76,70,34,63,62,10,60,120,58,120,109,112,109,101,116,97,32,120,0,6,214,198,231,51,167,131,210,118,22,70,246,38,83,166,231,51,166,214,87,70,18,242,114,7,131,167,134,215,7,70,179,210,117,132,213,2,7,70,246,246,198,182,151,66,3,34,227,146,227,18,211,19,50,194,6,103,32,0,97,109,101,119,111,114,107,32,49,46,54,39,62,10,60,114,100,102,58,82,68,70,32,120,109,108,110,115,58,114,100,102,61,39,104,116,116,112,58,47,47,119,119,119,46,119,51,46,111,114,103,47,49,57,57,57,47,0,3,3,34,243,35,34,215,38,70,98,215,55,150,231,70,23,130,214,231,50,50,114,7,134,214,198,231,51,166,149,131,210,118,135,71,71,3,162,242,246,231,50,230,22,70,246,38,82,230,54,246,210,246,149,130,243,18,224,0,48,47,39,62,10,60,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,32,114,100,102,58,97,98,111,117,116,61,39,102,98,102,100,55,57,55,99,45,54,54,57,56,45,49,49,101,48,45,48,48,48,48,45,0,3,35,83,150,67,35,131,131,35,67,99,83,34,114,7,134,214,198,231,51,167,6,70,99,210,118,135,71,71,3,162,242,246,231,50,230,22,70,246,38,82,230,54,246,210,247,6,70,98,243,18,227,50,242,115,227,199,0,0,100,102,58,80,114,111,100,117,99,101,114,62,71,80,76,32,71,104,111,115,116,115,99,114,105,112,116,32,56,46,55,48,60,47,112,100,102,58,80,114,111,100,117,99,101,114,62,10,60,112,100,102,58,75,101,121,119,0,6,247,38,71,51,226,130,147,194,247,6,70,99,164,182,87,151,118,247,38,71,51,224,163,194,247,38,70,99,164,70,87,54,55,38,151,7,70,150,246,227,224,163,199,38,70,99,164,70,87,54,55,38,151,7,70,150,240,0,110,32,114,100,102,58,97,98,111,117,116,61,39,102,98,102,100,55,57,55,99,45,54,54,57,56,45,49,49,101,48,45,48,48,48,48,45,50,53,57,100,50,56,56,50,52,54,53,50,39,32,120,109,108,110,115,58,0,7,134,215,3,210,118,135,71,71,3,162,242,246,231,50,230,22,70,246,38,82,230,54,246,210,247,134,23,2,243,18,227,2,242,115,227,199,134,215,3,164,214,246,70,150,103,148,70,23,70,83,227,35,3,19,18,211,0,0,52,45,49,49,84,49,53,58,49,51,58,50,50,43,48,50,58,48,48,60,47,120,109,112,58,77,111,100,105,102,121,68,97,116,101,62,10,60,120,109,112,58,67,114,101,97,116,101,68,97,116,101,62,50,48,49,49,0,2,211,3,66,211,19,21,67,19,83,163,19,51,163,35,34,179,3,35,163,3,3,194,247,134,215,3,164,55,38,86,23,70,84,70,23,70,83,224,163,199,134,215,3,164,55,38,86,23,70,247,37,70,246,246,195,229,192,0,51,55,54,92,51,55,55,92,48,48,48,80,92,48,48,48,68,92,48,48,48,70,92,48,48,48,67,92,48,48,48,114,92,48,48,48,101,92,48,48,48,97,92,48,48,48,116,92,48,48,48,111,92,48,48,48,114,0,5,195,3,3,2,5,195,3,3,5,101,195,3,3,6,85,195,3,3,7,37,195,3,3,7,53,195,3,3,6,149,195,3,3,6,245,195,3,3,6,229,195,3,3,2,5,195,3,3,3,21,195,3,3,2,229,195,0,0,48,48,48,92,48,48,48,46,92,48,48,48,50,60,47,120,109,112,58,67,114,101,97,116,111,114,84,111,111,108,62,60,47,114,100,102,58,68,101,115,99,114,105,112,116,105,111,110,62,10,60,114,100,102,58,68,101,0,7,54,55,38,151,7,70,150,246,226,7,38,70,99,166,22,38,247,87,67,210,118,102,38,102,67,115,147,118,50,211,99,99,147,130,211,19,22,83,2,211,3,3,3,2,211,35,83,150,67,35,131,131,35,67,99,83,32,0,39,32,120,109,108,110,115,58,120,97,112,77,77,61,39,104,116,116,112,58,47,47,110,115,46,97,100,111,98,101,46,99,111,109,47,120,97,112,47,49,46,48,47,109,109,47,39,32,120,97,112,77,77,58,68,111,99,0,7,86,214,86,231,68,148,67,210,118,102,38,102,67,115,147,118,50,211,99,99,147,130,211,19,22,83,2,211,3,3,3,2,211,35,83,150,67,35,131,131,35,67,99,83,34,114,243,224,163,199,38,70,99,164,70,87,48,0,99,114,105,112,116,105,111,110,32,114,100,102,58,97,98,111,117,116,61,39,102,98,102,100,55,57,55,99,45,54,54,57,56,45,49,49,101,48,45,48,48,48,48,45,50,53,57,100,50,56,56,50,52,54,53,50,39,0,2,7,134,214,198,231,51,166,70,51,210,118,135,71,71,3,162,242,247,7,87,38,194,230,247,38,114,246,70,50,246,86,198,86,214,86,231,71,50,243,18,227,18,242,114,6,70,51,166,102,247,38,214,23,67,210,118,16,0,112,112,108,105,99,97,116,105,111,110,47,112,100,102,39,62,60,100,99,58,116,105,116,108,101,62,60,114,100,102,58,65,108,116,62,60,114,100,102,58,108,105,32,120,109,108,58,108,97,110,103,61,39,120,45,100,101,0,6,102,23,86,199,66,115,229,195,51,115,101,195,51,115,117,195,3,3,4,69,195,3,3,6,245,195,3,3,6,53,195,3,3,7,85,195,3,3,6,213,195,3,3,6,85,195,3,3,6,229,195,3,3,7,69,195,0,0,48,48,49,60,47,114,100,102,58,108,105,62,60,47,114,100,102,58,65,108,116,62,60,47,100,99,58,116,105,116,108,101,62,60,100,99,58,99,114,101,97,116,111,114,62,60,114,100,102,58,83,101,113,62,60,114,100,0,6,99,166,198,147,229,195,51,115,101,195,51,115,117,195,3,3,7,53,195,3,3,6,85,195,3,3,6,197,195,3,3,7,101,195,3,3,6,213,195,3,3,6,21,195,3,3,7,69,195,3,3,7,67,194,247,38,64,0,102,58,108,105,62,60,47,114,100,102,58,83,101,113,62,60,47,100,99,58,99,114,101,97,116,111,114,62,60,100,99,58,100,101,115,99,114,105,112,116,105,111,110,62,60,114,100,102,58,83,101,113,62,60,114,100,102,0,3,166,198,147,226,130,147,194,247,38,70,99,166,198,147,227,194,247,38,70,99,165,54,87,19,227,194,246,70,51,166,70,87,54,55,38,151,7,70,150,246,227,227,194,247,38,70,99,164,70,87,54,55,38,151,7,70,144,0,111,110,62,10,60,47,114,100,102,58,82,68,70,62,10,60,47,120,58,120,109,112,109,101,116,97,62,10,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,0,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,0,162,2,2,2,2,2,2,2,2,2,2,2,2,0,0,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,32,0,2,2,0,163,195,247,135,6,22,54,182,87,66,6,86,230,67,210,119,114,115,243,224,166,86,230,71,55,71,38,86,22,208,166,86,230,70,246,38,160,163,34,3,2,6,246,38,160,163,195,194,245,7,38,246,71,86,48,0,101,114,40,71,80,76,32,71,104,111,115,116,115,99,114,105,112,116,32,56,46,55,48,41,10,47,67,114,101,97,116,105,111,110,68,97,116,101,40,68,58,50,48,49,49,48,52,49,49,49,53,49,51,50,50,43,48,0,3,34,115,3,2,114,144,162,244,214,246,68,70,23,70,82,132,67,163,35,3,19,19,3,67,19,19,19,83,19,51,35,34,179,3,34,115,3,2,114,144,162,245,70,151,70,198,82,133,195,51,115,101,195,51,115,117,192,0,48,48,48,68,92,48,48,48,111,92,48,48,48,99,92,48,48,48,117,92,48,48,48,109,92,48,48,48,101,92,48,48,48,110,92,48,48,48,116,92,48,48,48,49,41,10,47,67,114,101,97,116,111,114,40,92,51,0,3,115,101,195,51,115,117,195,3,3,5,5,195,3,3,4,69,195,3,3,4,101,195,3,3,4,53,195,3,3,7,37,195,3,3,6,85,195,3,3,6,21,195,3,3,7,69,195,3,3,6,245,195,3,3,7,37,192,0,48,48,48,32,92,48,48,48,86,92,48,48,48,101,92,48,48,48,114,92,48,48,48,115,92,48,48,48,105,92,48,48,48,111,92,48,48,48,110,92,48,48,48,32,92,48,48,48,49,92,48,48,48,46,92,48,48,0,3,3,5,195,3,3,2,229,195,3,3,3,34,144,162,244,23,87,70,134,247,34,133,195,51,115,101,195,51,115,117,195,3,3,7,53,195,3,3,6,85,195,3,3,6,197,195,3,3,7,101,195,3,3,6,213,195,0,0,48,48,97,92,48,48,48,116,92,48,48,48,116,41,10,47,75,101,121,119,111,114,100,115,40,41,10,47,83,117,98,106,101,99,116,40,41,62,62,101,110,100,111,98,106,10,120,114,101,102,10,48,32,49,48,10,48,0,3,3,3,3,3,3,3,3,3,2,3,99,83,83,51,82,6,98,0,163,3,3,3,3,3,3,3,67,35,82,3,3,3,3,3,2,6,226,0,163,3,3,3,3,3,3,35,51,115,114,3,3,3,3,3,2,6,224,0,32,10,48,48,48,48,48,48,48,51,54,54,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,50,50,53,32,48,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,48,49,53,32,48,48,48,48,0,3,2,6,226,0,163,3,3,3,3,3,3,3,35,3,98,3,3,3,3,3,2,6,226,0,163,3,3,3,3,3,3,3,83,19,130,3,3,3,3,3,2,6,226,0,163,3,3,3,3,3,3,3,67,131,146,3,0,0,48,48,48,48,32,110,32,10,48,48,48,48,48,48,48,53,56,52,32,48,48,48,48,48,32,110,32,10,116,114,97,105,108,101,114,10,60,60,32,47,83,105,122,101,32,49,48,32,47,82,111,111,116,32,49,32,48,0,2,5,34,2,244,150,230,102,242,3,34,3,2,5,32,162,244,148,66,5,179,195,131,83,68,67,68,35,52,19,35,131,147,99,4,35,19,67,67,35,99,4,51,4,99,132,68,67,147,3,99,67,51,115,227,195,131,80,0,52,68,52,66,51,65,50,56,57,54,48,66,49,52,52,50,54,48,67,48,70,56,68,68,57,48,54,52,51,55,62,93,10,62,62,10,115,116,97,114,116,120,114,101,102,10,50,55,56,50,10,37,37,69,79,70,10

    Matteo, .NET Developer
    Monday, April 11, 2011 1:42 PM
  • I used the above Base64 encoded string with both Convert.FromBase64String(...) and Base64Decoder methods and both gave me the same results i.e. a PDF with the word 'test' written inside it. Also, I got the same number of bytes from both "FromBase64String" and "GetDecoded" methods.
    -- Abhimanyu
    Tuesday, April 12, 2011 12:43 PM
  • It's possible that after decoding multiples of these large files that the Large Object Heap is hopelessly defragmented. Do you know what the applications Working Set is at the time of the OOM?
    Tuesday, April 12, 2011 8:45 PM
  • Hi Matteo,

     

    You may try to use FromBase64Transform method.

    The FromBase64String method is designed to process a single string that contains all the data to be decoded. To decode base-64 character data from a stream, use the System.Security.Cryptography.FromBase64Transform class.

     

    Have a nice day!


    Paul Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    • Proposed as answer by Albin Abel Wednesday, April 13, 2011 12:02 PM
    • Marked as answer by Paul Zhou Tuesday, April 26, 2011 3:07 AM
    Wednesday, April 13, 2011 8:11 AM
  • I used the above Base64 encoded string with both Convert.FromBase64String(...) and Base64Decoder methods and both gave me the same results i.e. a PDF with the word 'test' written inside it. Also, I got the same number of bytes from both "FromBase64String" and "GetDecoded" methods.
    -- Abhimanyu

    That is weird ! Let me try once again. If I don't get the same result, something's wrong...

    Matteo, .NET Developer
    Wednesday, April 13, 2011 11:44 AM
  • I used the above Base64 encoded string with both Convert.FromBase64String(...) and Base64Decoder methods and both gave me the same results i.e. a PDF with the word 'test' written inside it. Also, I got the same number of bytes from both "FromBase64String" and "GetDecoded" methods.
    -- Abhimanyu

    Hi Abhimanyu,

     

    There is something that I don't understand. You tried to decode my string and got the right result. I am still try to use the Code Project class to replace my old base64 encoding and decoding. For an unknown reason, it doesnt work.

     

    See my code below :

     

     

     /// <summary>
      /// Encodes data with base64 encoding.
      /// </summary>
      /// <param name="data">Data to encode.</param>
      /// <returns></returns>
      public static byte[] Base64Encode(byte[] data)
      {
       /*
        * 
        * 
        * Old way, this works when decoding with the old way too
    
       MemoryStream base64Data = null;
       MemoryStream retVal = null;
    
       try
       {
         base64Data = new MemoryStream(System.Text.Encoding.Default.GetBytes(Convert.ToBase64String(data)));
    
        // System encode won't split base64 data to lines, we need to do it manually
         retVal = new MemoryStream();
    
        while(true)
        {
         byte[] dataLine = new byte[76];
         int readedCount = base64Data.Read(dataLine, 0, dataLine.Length);
         // End of stream reached, end reading
         if(readedCount == 0)
          break;
    
         retVal.Write(dataLine, 0, readedCount);
         retVal.Write(new byte[] { (byte) '\r', (byte) '\n' }, 0, 2);
        }
    
        return retVal.ToArray();
       }
       catch
       {
        throw;
       }
       finally
       {
        if(base64Data != null)
         base64Data.Close();
    
        if(retVal != null)
         retVal.Close();
       }
        * 
        * 
        * */
    
       
       // New way, this doesn't work when decoding with the new way
       return System.Text.Encoding.Default.GetBytes(TheCodeProjectBase64Encoding.ToBase64(data));
      }
    
    
    
     /// <summary>
      /// Decodes base64 data.
      /// </summary>
      /// <param name="base64Data">Base64 decoded data.</param>
      /// <returns></returns>
      public static byte[] Base64Decode(byte[] data)
      {
    
      /*
       * 
       * Old way, this works when decoding data encoded with the old way !!
       * 
       * 
       * 
    
       string str = System.Text.Encoding.Default.GetString(data);
    
       if(str.Trim().Length > 0)
        return Convert.FromBase64String(str);
       else
        return new byte[] { };
       * 
       * */
    
       // New way, this doesn't work when decoding data encoded with the new way.
       return TheCodeProjectBase64Encoding.FromBase64(System.Text.Encoding.Default.GetString(data));
      }
    

     

    So, I don't know what appends... Actually, I am using open source code found here :

    http://www.codeproject.com/KB/cs/mime_project.aspx

    The encoding code is in the "Net_Core.cs" class.

    Because of the memory problem, I want to use my own base64 encoder. As you see in my above code, the result should be the same, regardless adding new lines to format the encoded data.

     

     


    Matteo, .NET Developer
    Monday, April 18, 2011 7:37 AM
  • Here is the code that I used:

          string s = "JVBERi0xLjQKJcfsj6IKNSAwIG9iago8PC9MZW5ndGggNiAwIFIvRmlsdGVyIC9GbGF0ZURlY29kZT4+CnN0cmVhbQp4nE2LsQoCQQxE+3xFyt3iYrK7d9m0ggh2SjqxOtRqi0P/H/cOFGcYePCYBZkkIa/9wtyA8dj3hAUq5TWb+Oe54d5hd1HsL38Ak5kV2ZygMlXUiamgN7iGdxwSyTSONdxjJi3hFQehlMz0J3PAePMTHBzOvR9KJB8fZW5kc3RyZWFtCmVuZG9iago2IDAgb2JqCjEyMQplbmRvYmoKNCAwIG9iago8PC9UeXBlL1BhZ2UvTWVkaWFCb3ggWzAgMCA1OTUgODQyXQovUm90YXRlIDAvUGFyZW50IDMgMCBSCi9SZXNvdXJjZXM8PC9Qcm9jU2V0Wy9QREYgL1RleHRdCi9Gb250IDggMCBSCj4+Ci9Db250ZW50cyA1IDAgUgo+PgplbmRvYmoKMyAwIG9iago8PCAvVHlwZSAvUGFnZXMgL0tpZHMgWwo0IDAgUgpdIC9Db3VudCAxCj4+CmVuZG9iagoxIDAgb2JqCjw8L1R5cGUgL0NhdGFsb2cgL1BhZ2VzIDMgMCBSCi9NZXRhZGF0YSA5IDAgUgo+PgplbmRvYmoKOCAwIG9iago8PC9SNwo3IDAgUj4+CmVuZG9iago3IDAgb2JqCjw8L0Jhc2VGb250L1RpbWVzLVJvbWFuL1R5cGUvRm9udAovU3VidHlwZS9UeXBlMT4+CmVuZG9iago5IDAgb2JqCjw8L1R5cGUvTWV0YWRhdGEKL1N1YnR5cGUvWE1ML0xlbmd0aCAxNzE3Pj5zdHJlYW0KPD94cGFja2V0IGJlZ2luPSfvu78nIGlkPSdXNU0wTXBDZWhpSHpyZVN6TlRjemtjOWQnPz4KPD9hZG9iZS14YXAtZmlsdGVycyBlc2M9IkNSTEYiPz4KPHg6eG1wbWV0YSB4bWxuczp4PSdhZG9iZTpuczptZXRhLycgeDp4bXB0az0nWE1QIHRvb2xraXQgMi45LjEtMTMsIGZyYW1ld29yayAxLjYnPgo8cmRmOlJERiB4bWxuczpyZGY9J2h0dHA6Ly93d3cudzMub3JnLzE5OTkvMDIvMjItcmRmLXN5bnRheC1ucyMnIHhtbG5zOmlYPSdodHRwOi8vbnMuYWRvYmUuY29tL2lYLzEuMC8nPgo8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0nZmJmZDc5N2MtNjY5OC0xMWUwLTAwMDAtMjU5ZDI4ODI0NjUyJyB4bWxuczpwZGY9J2h0dHA6Ly9ucy5hZG9iZS5jb20vcGRmLzEuMy8nPjxwZGY6UHJvZHVjZXI+R1BMIEdob3N0c2NyaXB0IDguNzA8L3BkZjpQcm9kdWNlcj4KPHBkZjpLZXl3b3Jkcz4oKTwvcGRmOktleXdvcmRzPgo8L3JkZjpEZXNjcmlwdGlvbj4KPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJvdXQ9J2ZiZmQ3OTdjLTY2OTgtMTFlMC0wMDAwLTI1OWQyODgyNDY1MicgeG1sbnM6eG1wPSdodHRwOi8vbnMuYWRvYmUuY29tL3hhcC8xLjAvJz48eG1wOk1vZGlmeURhdGU+MjAxMS0wNC0xMVQxNToxMzoyMiswMjowMDwveG1wOk1vZGlmeURhdGU+Cjx4bXA6Q3JlYXRlRGF0ZT4yMDExLTA0LTExVDE1OjEzOjIyKzAyOjAwPC94bXA6Q3JlYXRlRGF0ZT4KPHhtcDpDcmVhdG9yVG9vbD5cMzc2XDM3N1wwMDBQXDAwMERcMDAwRlwwMDBDXDAwMHJcMDAwZVwwMDBhXDAwMHRcMDAwb1wwMDByXDAwMCBcMDAwVlwwMDBlXDAwMHJcMDAwc1wwMDBpXDAwMG9cMDAwblwwMDAgXDAwMDFcMDAwLlwwMDAwXDAwMC5cMDAwMjwveG1wOkNyZWF0b3JUb29sPjwvcmRmOkRlc2NyaXB0aW9uPgo8cmRmOkRlc2NyaXB0aW9uIHJkZjphYm91dD0nZmJmZDc5N2MtNjY5OC0xMWUwLTAwMDAtMjU5ZDI4ODI0NjUyJyB4bWxuczp4YXBNTT0naHR0cDovL25zLmFkb2JlLmNvbS94YXAvMS4wL21tLycgeGFwTU06RG9jdW1lbnRJRD0nZmJmZDc5N2MtNjY5OC0xMWUwLTAwMDAtMjU5ZDI4ODI0NjUyJy8+CjxyZGY6RGVzY3JpcHRpb24gcmRmOmFib3V0PSdmYmZkNzk3Yy02Njk4LTExZTAtMDAwMC0yNTlkMjg4MjQ2NTInIHhtbG5zOmRjPSdodHRwOi8vcHVybC5vcmcvZGMvZWxlbWVudHMvMS4xLycgZGM6Zm9ybWF0PSdhcHBsaWNhdGlvbi9wZGYnPjxkYzp0aXRsZT48cmRmOkFsdD48cmRmOmxpIHhtbDpsYW5nPSd4LWRlZmF1bHQnPlwzNzZcMzc3XDAwMERcMDAwb1wwMDBjXDAwMHVcMDAwbVwwMDBlXDAwMG5cMDAwdFwwMDAxPC9yZGY6bGk+PC9yZGY6QWx0PjwvZGM6dGl0bGU+PGRjOmNyZWF0b3I+PHJkZjpTZXE+PHJkZjpsaT5cMzc2XDM3N1wwMDBzXDAwMGVcMDAwbFwwMDB2XDAwMG1cMDAwYVwwMDB0XDAwMHQ8L3JkZjpsaT48L3JkZjpTZXE+PC9kYzpjcmVhdG9yPjxkYzpkZXNjcmlwdGlvbj48cmRmOlNlcT48cmRmOmxpPigpPC9yZGY6bGk+PC9yZGY6U2VxPjwvZGM6ZGVzY3JpcHRpb24+PC9yZGY6RGVzY3JpcHRpb24+CjwvcmRmOlJERj4KPC94OnhtcG1ldGE+CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAKPD94cGFja2V0IGVuZD0ndyc/PgplbmRzdHJlYW0KZW5kb2JqCjIgMCBvYmoKPDwvUHJvZHVjZXIoR1BMIEdob3N0c2NyaXB0IDguNzApCi9DcmVhdGlvbkRhdGUoRDoyMDExMDQxMTE1MTMyMiswMicwMCcpCi9Nb2REYXRlKEQ6MjAxMTA0MTExNTEzMjIrMDInMDAnKQovVGl0bGUoXDM3NlwzNzdcMDAwRFwwMDBvXDAwMGNcMDAwdVwwMDBtXDAwMGVcMDAwblwwMDB0XDAwMDEpCi9DcmVhdG9yKFwzNzZcMzc3XDAwMFBcMDAwRFwwMDBGXDAwMENcMDAwclwwMDBlXDAwMGFcMDAwdFwwMDBvXDAwMHJcMDAwIFwwMDBWXDAwMGVcMDAwclwwMDBzXDAwMGlcMDAwb1wwMDBuXDAwMCBcMDAwMVwwMDAuXDAwMDBcMDAwLlwwMDAyKQovQXV0aG9yKFwzNzZcMzc3XDAwMHNcMDAwZVwwMDBsXDAwMHZcMDAwbVwwMDBhXDAwMHRcMDAwdCkKL0tleXdvcmRzKCkKL1N1YmplY3QoKT4+ZW5kb2JqCnhyZWYKMCAxMAowMDAwMDAwMDAwIDY1NTM1IGYgCjAwMDAwMDA0MjUgMDAwMDAgbiAKMDAwMDAwMjM3NyAwMDAwMCBuIAowMDAwMDAwMzY2IDAwMDAwIG4gCjAwMDAwMDAyMjUgMDAwMDAgbiAKMDAwMDAwMDAxNSAwMDAwMCBuIAowMDAwMDAwMjA2IDAwMDAwIG4gCjAwMDAwMDA1MTggMDAwMDAgbiAKMDAwMDAwMDQ4OSAwMDAwMCBuIAowMDAwMDAwNTg0IDAwMDAwIG4gCnRyYWlsZXIKPDwgL1NpemUgMTAgL1Jvb3QgMSAwIFIgL0luZm8gMiAwIFIKL0lEIFs8ODU0RDRCM0EyODk2MEIxNDQyNjBDMEY4REQ5MDY0Mzc+PDg1NEQ0QjNBMjg5NjBCMTQ0MjYwQzBGOEREOTA2NDM3Pl0KPj4Kc3RhcnR4cmVmCjI3ODIKJSVFT0YK";
          using (FileStream sw = new FileStream(@"c:\testdata\test.pdf", FileMode.Create))
          {
            /////// Test 1 ///////
            //TestBase64.Base64Decoder d = new TestBase64.Base64Decoder(s.ToCharArray());
            //byte[] buffer = d.GetDecoded();
            //////////////////////
    
            /////// Test 2 ///////
            byte[] buffer = Convert.FromBase64String(s);
            sw.Write(buffer, 0, buffer.Length);
            //////////////////////
          }
    
    Include the Base64Encoder and Base64Decoder classes in the project. The string variable 's' contains your main encoded string.


    -- Abhimanyu
    Monday, April 18, 2011 3:39 PM
  • Thanks Abhimanyu for your code snippet,

     

    I think that the difference between your code and mine is that mine contains new lines chars "\r\n". That seems not to be a problem with the Convert.FromBase64String(...) but it is for the TestBase64.Base64Decoder(...) one. In my environment, the encoded strings come from MIME files, that are formated with new lines. I don't understand why it works with the .NET class and not the Code Project one, considering that base64 is a standard.

     

    To illustrate it, I made a small .NET sample code that you can find here :

     

    http://www.megafileupload.com/en/file/309891/TestBase64-zip.html

     

    Best regards.

     

    Matteo


    Matteo, .NET Developer
    Tuesday, April 19, 2011 8:28 AM
  • Hi, Here I am !

    I have made some changes to the base64 encoder from Code Project. Actually, if I get rid of all the special chars, \r, \n, \t, it works fine. I guess it's what the Framework class does.

    But I still have a System.OutOfMemoryException, when working with a large amount of data. This is the code that throws the Exception :

     private char[] DeleteUnvalidChars(char[] input)
     {
     for(int i = 0; i < MimeBase64Encoding.unvalidChars.Length; i++)
     if(Array.IndexOf(input, MimeBase64Encoding.unvalidChars[i]) >= 0)
      input = new string(input).Replace(Convert.ToString(MimeBase64Encoding.unvalidChars[i]), string.Empty).ToCharArray(); // THROWS OUT OF MEMORY EXCEPTION !!!
    
     return input;
     }
    

    Of course, the char[] input has a large amount of data ({char[75155112]}). Actually, I could modify my code to consume less memory, but I am affraid it will bug anyway if the amount of data is too large.

    I am not really use to work with large amount of data. Can anyone give me an advice of what I can do to avoid a System.OutOfMemoryException to be thrown ?

    Actually, in my case, this System.OutOfMemoryException appends because of those reasons :

    - A Stream is opened from a file, or a MemoryStream from a byte array

    - The stream is parsed and encoded, by other objects, that have bytes and chars array within

    - The stream closes at the end

    What gives : MemoryStream (byte[]) +  Base64Encoding (internal chars[] and bytes[]) + internal transformations (new string[]{})

    With all that stuff, I guess my ({char[75155112]} is multiplicated by 4, maybe more. So I use 300 MB of memory only to encode or decode one file content. Having a Stream opened from à 50 MB file, and doing encoding operations consumes a lot of memory. On a 32 bits machine, I guess there's not enough memory to manage all that stuff.

    My problem is that I have no control on the whole process, because many objects are used, havin their own internal memory aspect.

    I have the idea of using a file buffer on the hard disk for the base64 encoding operations, because it's my own code now. But if there is a better way to deal with my System.OutOfMemoryException problem it would be better.

    Best regards.

    Matteo


    Matteo, .NET Developer

    Tuesday, April 19, 2011 11:33 AM
  • The better way to avoid this would be to not create such a large String. To have that code use less memory my first attempt in changing the code would be to iterate over input, and count the number of invalid chars. Create a new array the size of the valid chars remaining, then iterate over input again, and only copy valid chars to the new array.
    Tuesday, April 19, 2011 2:09 PM
  • The better way to avoid this would be to not create such a large String. To have that code use less memory my first attempt in changing the code would be to iterate over input, and count the number of invalid chars. Create a new array the size of the valid chars remaining, then iterate over input again, and only copy valid chars to the new array.


    Hi Jader,

     

    I tried, but the performances are bad. The method takes a long time :

     

     private char[] DeleteUnvalidChars(char[] input)
          {
            // THIS CODE TAKES A LONG TIME, CANNOT USE IT !!!
            List<int> unvalidCharsIndexes = new List<int>();
            int lastIndex;
            int i;
            int compt;
    
            for(i = 0; i < MimeBase64Encoding.unvalidChars.Length; i++)
            {
              lastIndex = 0;
    
              while(lastIndex >= 0)
              {
                lastIndex = Array.IndexOf(input, MimeBase64Encoding.unvalidChars[i], lastIndex);
    
                if(lastIndex >= 0)
                {
                  unvalidCharsIndexes.Add(lastIndex);
    
                  if(lastIndex < (input.Length - 1))
                    lastIndex++;
                  else
                    lastIndex = -1;
                }
              }
            }
    
            if(input.Length - unvalidCharsIndexes.Count == 0)
              return new char[0];
    
            char[] ret = new char[input.Length - unvalidCharsIndexes.Count];
    
            compt = 0;
    
            for(i = 0; i < input.Length; i++)
            {
              if(unvalidCharsIndexes.IndexOf(i) >= 0)
                continue;
    
              ret[compt] = input[i];
              compt++;
            }
    
            return ret;
    
            // THIS CODE IS MUCH FASTER !!!
    
            //for(int i = 0; i < MimeBase64Encoding.unvalidChars.Length; i++)
            //  if(Array.IndexOf(input, MimeBase64Encoding.unvalidChars[i]) >= 0)
            //    input = new string(input).Replace(Convert.ToString(MimeBase64Encoding.unvalidChars[i]), string.Empty).ToCharArray();
    
            //return input;
          }

     


    Matteo, .NET Developer
    Tuesday, April 19, 2011 3:30 PM
  • I was thinking of something more like

    private char[] RemoveInvalidChars(char[] input)
    {
    int numberOfValid = 0;
    for (int i = 0; i < input.Length; i++)
    {
      if (!IsInvalidChar(input[i]))
      {
        numberOfValid++;
      }
    }
    char[] validChars = new char[numberOfValid];
    for (int i = 0, n = 0; i < input.Length; i++)
    {
      if (!IsInvalidChar(input[i]))
      {
        validChars[n] = input[i];
        n++;
      }
    }
    
    return validChars;
    }
    

    Tuesday, April 19, 2011 3:51 PM
  • I was thinking of something more like

     

    private char[] RemoveInvalidChars(char[] input)
    {
    int numberOfValid = 0;
    for (int i = 0; i < input.Length; i++)
    {
     if (!IsInvalidChar(input[i]))
     {
      numberOfValid++;
     }
    }
    char[] validChars = new char[numberOfValid];
    for (int i = 0, n = 0; i < input.Length; i++)
    {
     if (!IsInvalidChar(input[i]))
     {
      validChars[n] = input[i];
      n++;
     }
    }
    
    return validChars;
    }
    

     

    Thanks Jader, your code runs faster than mine ;)

     

    But it still consume a lot of memory... Even if I cannot do better...


    Matteo, .NET Developer
    Tuesday, April 19, 2011 4:26 PM
  • If you're willing to modify the input you can do

    private int RemoveInvalidChars(char[] input)
    {
    int index = 0;
    for (int i = 0; i < input.Length; i++)
    {
      if (!IsInvalidChar(input[i]))
      {
        input[index] = input[i];
        index++;
      }
    }
    for (int j = index; j < input.Length; j++)
    {
      input[j] = Char.MinValue;
    }
    return index;
    }
    

    • Marked as answer by Paul Zhou Tuesday, April 26, 2011 3:07 AM
    Tuesday, April 19, 2011 5:03 PM
  • Check out this article, which explains how to implement your own Encoder class which you can plug into the standard Stream Reader/Writer classes of the BCL:

    http://blogs.microsoft.co.il/blogs/kim/archive/2007/10/09/base64-encode-large-files-very-large-files.aspx

    It still have a few things that aren't working fully, but it is a very nice place to start.

    • Marked as answer by Paul Zhou Tuesday, April 26, 2011 3:07 AM
    Tuesday, April 19, 2011 5:13 PM
  • Thanks Jesse !

     

    Based on your article, I decided to modify the encoder avaliable on The Code Project. To avoid having System.OutOfMemoryException thrown, I use file streams instead of the RAM.

     

    I published the code so that the developer can use it if he wants. See the code on The Code Project artcile page, under "Comments and discussions" sections.

     

    http://www.codeproject.com/KB/cs/base64encdec.aspx?msg=3870543#xx3870543xx

     

    I hope this will help people having the same issue.

     

    Best regards.


    Matteo, .NET Developer
    • Marked as answer by joujoukinder Wednesday, April 27, 2011 2:21 PM
    Wednesday, April 27, 2011 2:21 PM