none
How much increases our application memory size when we use Pinvoke to call functions of an unmanaged dll ? RRS feed

  • Question

  • Hi to all,
    this may sound like a stupid question, but I've not been able to find any answer on the web.
    Supposing that in my C# application there are declared several functions of the unmanaged DLL "A".  Disk size of dll "A" is 5 mb and, based on the called function, this dll can also call DLL "B" which a disk size of 50 mb.

    Correct me if I'm wrong:
    as long as I don't call any of the unmanaged functions, the total memory usage of my program can be considered the same as if it was being compiled without any reference to the dll.
    As soon as any function of DLL "A" is being called, the memory usage of my program increase immediately of 5Mb and, in the case that dll "A" calls a function of dll "B", the memory usage will increase of 50 additional megabytes. 
    It doesn't matter if the first function called in dll "A" only returns a simple boolean value and it doesn't matter if the job performed from the unmanaged side is short (for example, like executing two lines of code): as soon as the functions are called, the involved dlls will become parts of my program and their memory usage will stay alive until the program terminates.

    Probably my interpretation is wrong and I would really appreciate if someone could clarify how understand the real memory usage.

    Thanks
    Tuesday, April 24, 2012 7:23 PM

Answers

  • Hi Mark,

    In general calling a method in an unmanaged DLL should not pull the full DLL into the private working set of your application. This can happen if the native DLL gets relocated due to a base address collision. This is not very likely on an OS of Vista or later, but it is still possible.

    You can use VMMap (link) to look at the working set and private working set breakdown of your process before and after your function call and confirm that the entire DLL is in fact being read into private memory, and if not, what other type of memory is being used.

    • Marked as answer by mark_555 Thursday, May 3, 2012 12:40 PM
    Friday, April 27, 2012 9:25 PM

All replies

  • Memory usage of a program can variy depending how and where the memory is declared.  It doesn't make any difference if you are calling a dll or calling a function inside you program.

    If you make a seperate class for calling the DLL and destroy the class when the dll completes excution then any memory space used in the class will also be released.

    See this webpage for Pinvoke.  The call to the DLL is inside a class.

    http://msdn.microsoft.com/en-us/library/aa446536.aspx


    jdweng

    • Proposed as answer by Mike FengModerator Thursday, April 26, 2012 7:37 AM
    • Unproposed as answer by mark_555 Friday, April 27, 2012 2:38 PM
    Tuesday, April 24, 2012 7:53 PM
  • Hi Joel and thanks for your answer.
    I think you're talking about the additional memory that the unmanaged Dll can allocate on runtime using common functions like malloc, new, alloc and so on, and that's a different story.
    As you said, memory can be allocated and released using different methods, but this is not the kind of memory I'm talking about.

    When from my code I call an unmanaged function for the first time, I noticed that execution is pretty slow while all successive calls are much faster. I didn't notice any difference in terms of speed between a function executed inside my application and another function executed in the managed dll.

    I suppose this happen because the CLR, after the first call to the unmanaged function, fully loads the dll in memory. This memory can be considered as part of the program and will be released only once the program terminates.

    In my environment directory there's an unmanaged dll with a disk size of 1.3Mb and my program refers to a few functions of this dll. With Windows Task Manager on hands, I can see that memory usage of my program (private working set) is 6mb. As soon as the first unmanaged function is called, the reported memory usage increases immediately to 7.3Mb, which corresponds exactly to the disk size of the involved dll. Furthermore, for every additional call (which just returns a short string) the memory usage increases by 5-10Kb (Yes, the memory allocated in the unmanaged side is released every time).
    After calling the same function 10/15 times, the reported memory usage does not increase anymore for the subsequent calls.

    I don't understand this strange behavior, but at this point I think that I was not wrong on my originary interpretation regarding the memory usage.

    I hope that someone can tell me that I'm totally wrong, but from the above, appear to be that even for a single call to an unmanaged function, we will have to take into account that the total memory usage of our program will increase of the same size that the dll has on disk. In addition to that, this memory will only be released when the program terminates.
    Friday, April 27, 2012 2:38 PM
  • Where is the string in memory.  I assume pass a pointer to the string when you call the dll and the memory for the pointer in inside your class code.  I was looking at some code I copied from the web a couple of years ago that I've been updating.  I'm wondering if this code behaves the same as your code.  Maybe you want to experiment with this code to see if you get the same behavior.

           // The max number of physical addresses. 
            const int MAXLEN_PHYSADDR = 8;
           // Define the MIB_IPNETROW structure. 
          
            struct MIB_IPNETROW 
            {
                public int dwIndex;
                public int dwPhysAddrLen;
                public byte mac0;
                public byte mac1;
                public byte mac2;
                public byte mac3;
                public byte mac4;
                public byte mac5;
                public byte mac6;
                public byte mac7;
                public int dwAddr;
                public int dwType;
            }
            // Declare the GetIpNetTable function.
            [DllImport("IpHlpApi.dll")]
            [return: MarshalAs(UnmanagedType.U4)]
            static extern int GetIpNetTable(
               IntPtr pIpNetTable,
               [MarshalAs(UnmanagedType.U4)] 
             ref int pdwSize,
               bool bOrder);
            // The insufficient buffer error. 
            const int ERROR_INSUFFICIENT_BUFFER = 122;
            static IntPtr buffer;
            static int result;
            public GetArpTable()
            {
                // The number of bytes needed. 
                int bytesNeeded = 0;
                // The result from the API call. 
                result = GetIpNetTable(IntPtr.Zero, ref bytesNeeded, false);
                // Call the function, expecting an insufficient buffer. 
                if (result != ERROR_INSUFFICIENT_BUFFER)
                {
                    // Throw an exception. 
                    throw new Win32Exception(result);
                }
                // Allocate the memory, do it in a try/finally block, to ensure 
                // that it is released. 
                buffer = IntPtr.Zero;
                // Try/finally. 
                try
                {
                    // Allocate the memory. 
                    buffer = Marshal.AllocCoTaskMem(bytesNeeded);
                    // Make the call again. If it did not succeed, then 
                    // raise an error. 
                    result = GetIpNetTable(buffer, ref bytesNeeded, false);
                    // If the result is not 0 (no error), then throw an exception. 
                    if (result != 0)
                    {
                        // Throw an exception. 
                        throw new Win32Exception(result);
                    }
                }
                finally
                {
                   
                }
              }


    jdweng

    Friday, April 27, 2012 3:26 PM
  • Hi Mark,

    In general calling a method in an unmanaged DLL should not pull the full DLL into the private working set of your application. This can happen if the native DLL gets relocated due to a base address collision. This is not very likely on an OS of Vista or later, but it is still possible.

    You can use VMMap (link) to look at the working set and private working set breakdown of your process before and after your function call and confirm that the entire DLL is in fact being read into private memory, and if not, what other type of memory is being used.

    • Marked as answer by mark_555 Thursday, May 3, 2012 12:40 PM
    Friday, April 27, 2012 9:25 PM
  • Hi Mark,

    Loading the DLL into memory and memory involved in creating objects are different things. Loading the DLL into memory takes advantage of Paging. By this, only the code that needs to be executed at any time (Or code that is frequently used) resides on primary memory which remainind DLL will reside in Pagefile. It means, when DLL is loaded, it doesn't mean that entire DLL should be present on main memory.

    On the other hand, you say calling unmanaged method which returns a string increases memory by 5-10Kb. That is because, the memory occupied includes size of the string, size of the interop objects etc.

    I hope this helps.


    Please mark this post as answer if it solved your problem. Happy Programming!

    Monday, April 30, 2012 5:47 AM
  • Hi joel and thanks for your help. I experienced the same behaviour with your code. I have performed the test using the compiled assembly. The first unmanaged call caused an immediate increase of the private working set (around 80Kb which approximately corresponds to the size of IpHlpApi.dll). Everytime that GetArpTable() is called, the used memory increases of 10-20Kb and after 10 calls doesn't increase anymore. This also happens even after releasing the memory with Marshal.FreeCoTaskMem(buffer).
    Thanks
    Wednesday, May 2, 2012 6:02 PM
  • Yo mean I had a memory leak that I didn't know about?  See if dispose solves the problem. (see webpage below).  I found this article over the weekend.  Another posting had problems using SMTP protocol.  The interface wasn't closing and he added one line to the code to solve problem. 

    http://msdn.microsoft.com/en-us/library/system.idisposable.dispose.aspx


    jdweng

    Wednesday, May 2, 2012 6:09 PM
  • Hi Tan and thanks for your suggestion. I'm not sure on how to interpreting the tool you suggested me, but looking into the "Image" section I've discovered that when the first unmanaged call is done, the same dll, automatically calls other ten dlls (which are part of the same environment). Since the sum of these dlls is more than 50 Mb, I deduce that they are not loaded in memory because the private working set only increases of 1Mb.
    The program does not frees this memory usage even when the api returns. I can see a little increase of memory usage for every call until, after 10-15 calls, the reported usage doesn't increase anymore for every subsequent call. If I continue to call the functions after leaving the program idle for a few minutes, the memory usage continue to increase. The memory usage randomly decreases and independently from whatever I do in the program,  seems that it never exceeds a certain size.

    I'm not sure, can I consider this as the normal behavior or there's something else I should check?

    Thanks
    Wednesday, May 2, 2012 7:04 PM
  • Hi Adavesh and thanks for your comment. I have performed other tests and updated the thread with new comments.
    Yes, calling unmanaged method which returns a string, increases memory by 5-10Kb , even if I release the allocated memory when the api returns.
    So, it's not just the size of the string, but also the interop objects  automatically created from the CLR. 
    Correct me if I'm wrong, at this point, my main job it's just the one to release memory allocated from the unmanaged side and the CLR will do the rest like destroying all the interop objects he created to exchange the data. Thanks
    Wednesday, May 2, 2012 7:16 PM
  • Yes, the code posted here clearly shows a memory leak. You've allocated a block of memory without releasing it. You should call "Marshal.FreeCoTaskMem(buffer)" before "GetArpTable" returns.
    I don't think that IDisposable interface is required for your sample code. A few weeks ago, I opened a thread regarding the real advantages of this method. It turned out that, IN MY PERSONAL SITUATION, it was just an useless overload of code. Take a look on my thread, you should find usefull infos.
    Wednesday, May 2, 2012 7:42 PM
  • Hi Adavesh and thanks for your comment. I have performed other tests and updated the thread with new comments.
    Yes, calling unmanaged method which returns a string, increases memory by 5-10Kb , even if I release the allocated memory when the api returns.
    So, it's not just the size of the string, but also the interop objects  automatically created from the CLR. 
    Correct me if I'm wrong, at this point, my main job it's just the one to release memory allocated from the unmanaged side and the CLR will do the rest like destroying all the interop objects he created to exchange the data. Thanks

    Correct.

    Please mark this post as answer if it solved your problem. Happy Programming!

    Thursday, May 3, 2012 2:18 AM
  • Great. Seems to be that there was nothing wrong regarding the memory usage of my program and all of what I was experiencing was nothing else than the normal behavior. Thanks
    Thursday, May 3, 2012 12:29 PM
  • All the comments from this thread helped me to understand the situation, but only thanks to your answer I really found that the increase of 1Mb after the first call to the unmanaged method was not due to the full dll pulled into the private working set but to the fact that this dll was calling other 10 different dlls.

    Thanks
    Thursday, May 3, 2012 12:40 PM