none
Kernel memory allocations in C++

    Question

  • Hello,

     

    While attempted usage of operator 'new' I got a compilation error due to usage of exceptions with "\kernel" switch (once I removed the inclusion of <new> header I got linkage error). Is it possible to use the standard 'new' operator in VS2011 kernel mode driver (e.g., by disabling exceptions) or is it totally restricted now?

    Tuesday, November 22, 2011 10:20 AM

Answers

  • operator new() that throws is now banned.  practically speaking, i have no idea how you got the std throwing operator new to work before /kernel was added either since it didn't use a kernel memory allocation function for the underlying alloc.  just define your own global operator new that doesn't throw if you want something equivalent and then check for NULL on return from the call
    d -- This posting is provided "AS IS" with no warranties, and confers no rights.
    • Marked as answer by ice2dude Tuesday, November 22, 2011 6:01 PM
    Tuesday, November 22, 2011 5:26 PM
  • You have many options for allocations in the kernel: ExAllocatePoolWithTag, ExAllocatePoolWithTagPriority, ExAllocateFromNPagedLookasideList, ExAllocateFromPagedLookasideList, etc. Pick what better fits your task.  C++ operator new is not among these options though. Unless you hack your own, using the mentioned kernel APIs.

    -- pa

     


    • Edited by Pavel A Tuesday, November 22, 2011 1:04 PM
    • Marked as answer by ice2dude Tuesday, November 22, 2011 6:01 PM
    Tuesday, November 22, 2011 1:02 PM

All replies

  • You have many options for allocations in the kernel: ExAllocatePoolWithTag, ExAllocatePoolWithTagPriority, ExAllocateFromNPagedLookasideList, ExAllocateFromPagedLookasideList, etc. Pick what better fits your task.  C++ operator new is not among these options though. Unless you hack your own, using the mentioned kernel APIs.

    -- pa

     


    • Edited by Pavel A Tuesday, November 22, 2011 1:04 PM
    • Marked as answer by ice2dude Tuesday, November 22, 2011 6:01 PM
    Tuesday, November 22, 2011 1:02 PM
  • Yes, I'm aware of these. Just wondered about the new operator - it wasn't previously banned from kernel and possible implications of exceptions where developers' responsibility, right?
    Tuesday, November 22, 2011 1:11 PM
  • Yes, I'm aware of these. Just wondered about the new operator - it wasn't previously banned from kernel and possible implications of exceptions where developers' responsibility, right?

    Right - just like with, say, printf. No one banned using printf in kernel, but once you manage to get it there :) it's your responsibility to handle implications.

    -- pa

    Tuesday, November 22, 2011 5:18 PM
  • operator new() that throws is now banned.  practically speaking, i have no idea how you got the std throwing operator new to work before /kernel was added either since it didn't use a kernel memory allocation function for the underlying alloc.  just define your own global operator new that doesn't throw if you want something equivalent and then check for NULL on return from the call
    d -- This posting is provided "AS IS" with no warranties, and confers no rights.
    • Marked as answer by ice2dude Tuesday, November 22, 2011 6:01 PM
    Tuesday, November 22, 2011 5:26 PM
  • operator new() that throws is now banned.  practically speaking, i have no idea how you got the std throwing operator new to work before /kernel was added either since it didn't use a kernel memory allocation function for the underlying alloc.  just define your own global operator new that doesn't throw if you want something equivalent and then check for NULL on return from the call
    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    That's exactly what I intend to do. I'm starting to develop on VS11 and as a first step just wanted the new code to compile without any intention to execute. The compiler had surprised me a bit with the error so I wanted to be sure that this is the expected behavior for operator new() in kernel (since it wasn't like that with previous compiler versions) and not something related to wrong environment configuration of the VS11 on my machine or some missing kernel libraries.

    Thanks.

     

    P.S. Many of us developers and sw architects outthere would benefit a lot if MS updates the C++ for Kernel Mode Drivers white paper.

    Tuesday, November 22, 2011 6:00 PM
  • Another question regarding memory allocations: I'm a bit confused regarding the NdisAllocateMemoryWithTag and NdisAllocateMemoryWithTagPriority functions. The NdisAllocateMemoryWithTag documentation seems to be in a need of editing. Quote: "Not supported for NDIS 6.0 drivers in Windows Vista. NdisAllocateMemoryWithTagPriority. Supported for NDIS 5.1 drivers in Windows Vista and Windows XP". So is it supported or not? I've looked into the ndis.h and it seems to be very much enabled for NDIS 6. I intend to overload the global new operator and would happily use NdisAllocateMemoryWithTagPriority but it requires ndis handle parameter to be passed - and that's something I don't want to do in order to keep the original new operator signature.
    Wednesday, November 23, 2011 11:15 PM
  • ice2dude wrote:
    >
    >Yes, I'm aware of these. Just wondered about the new operator - it wasn't
    >previously banned from kernel and possible implications of exceptions
    >where developers' responsibility, right?
     
    Well, it has never been banned, but there is no default implementation.  As
    long as you supply your own "operator new" and "operator delete" that does
    the right thing, it works fine.
    --
    Tim Roberts, timr@probo.com
    Providenza & Boekelheide, Inc.
     

    Tim Roberts, VC++ MVP Providenza & Boekelheide, Inc.
    Thursday, November 24, 2011 5:53 AM
  • ice2dude wrote:
    >
    >Yes, I'm aware of these. Just wondered about the new operator - it wasn't
    >previously banned from kernel ...
     
    And, by the way, the WDK include file <kcom.h> includes definitions for
    operator new and delete that solve the problem nicely.
    --
    Tim Roberts, timr@probo.com
    Providenza & Boekelheide, Inc.
     

    Tim Roberts, VC++ MVP Providenza & Boekelheide, Inc.
    Thursday, November 24, 2011 5:53 AM
  • ice2dude wrote:
    >
    >Yes, I'm aware of these. Just wondered about the new operator - it wasn't
    >previously banned from kernel ...
     
    And, by the way, the WDK include file <kcom.h> includes definitions for
    operator new and delete that solve the problem nicely.
    --
    Tim Roberts, timr@probo.com
    Providenza & Boekelheide, Inc.
     

    Tim Roberts, VC++ MVP Providenza & Boekelheide, Inc.
    Thank you Tim, these are interesting bits of info. Not sure I'll use this implementation but still nice to know. Can you please comment on my second question?
    Thursday, November 24, 2011 10:06 AM
  • Another question regarding memory allocations: I'm a bit confused regarding the NdisAllocateMemoryWithTag and NdisAllocateMemoryWithTagPriority functions. The NdisAllocateMemoryWithTag documentation seems to be in a need of editing. Quote: "Not supported for NDIS 6.0 drivers in Windows Vista. NdisAllocateMemoryWithTagPriority. Supported for NDIS 5.1 drivers in Windows Vista and Windows XP". So is it supported or not? I've looked into the ndis.h and it seems to be very much enabled for NDIS 6.


    It is *deprecated* for NDIS6. Even if it still exists in ndis.h and you can use it, better don't.

    Note that the significant difference between NdisAllocateMemoryWithTagPriority and NdisAllocateMemoryWithTag is that the latter returns a pointer, which is more convenient. The priority value you want to use is probably NormalPoolPriority, and C++ allows to specify a default value for a parameter.

    I intend to overload the global new operator and would happily use NdisAllocateMemoryWithTagPriority but it requires ndis handle parameter to be passed - and that's something I don't want to do in order to keep the original new operator signature.
    The handle is the one of your miniport or protocol or other NDIS creature. It is "global" in the scope of your driver. You don't mean to create a really  global C++ operator new for the rest of the kernel.

    -- pa


    • Edited by Pavel A Thursday, November 24, 2011 6:16 PM
    Thursday, November 24, 2011 6:10 PM
  • The handle is the one of your miniport or protocol or other NDIS creature. It is "global" in the scope of your driver. You don't mean to create a really  global C++ operator new for the rest of the kernel.

    -- pa


    I need to support several NICs with single miniport driver. Correct me if I'm wrong, but if I keep the handler as global variable in kernel it will be shared among all loaded instances of the driver. So the choices I see are either to create a global new() which accepts the driver handler as one of its parameters or to define the operator new() in the miniport context  which I allocate during initialization (and the handler will be equal to 'this' in this case). This stuff is kind of uncharted waters for me so I'll appreciate any insights on the subject.
    Thursday, November 24, 2011 7:21 PM
  • The handle of the driver, received from NdisRegisterProtocolDriver() is valid for all "device" (in NDIS sense of it)  instances in this driver. There is only one instance of the miniport *driver* itself. It is a singleton.

    Also, why you are so obsessed with operator new. Things can be created without new. Make a factory which internally uses native NdisAllocate.... and returns pointer?

    -- pa

    Thursday, November 24, 2011 10:19 PM
  • The handle of the driver, received from NdisRegisterProtocolDriver() is valid for all "device" (in NDIS sense of it)  instances in this driver. There is only one instance of the miniport *driver* itself. It is a singleton.

    I have a miniport driver, not protocol - so it's NdisMRegisterMiniportDriver (suppose that doesn't change the answer). I want to understand a bit more regarding the driver instances: suppose I have 2 NICs that I want to manage with the same driver. Will the device manager gui display them as 2 devices? Will the OS call the DriverEntry() twice, each time for a different device? How can I know the target device for OS call (e.g. the OS calls the miniport send packets handler - how will I know to which NIC to route the request?) Sorry for all these questions, they probably seem naive :)

    Also, why you are so obsessed with operator new. Things can be created without new. Make a factory which internally uses native NdisAllocate.... and returns pointer?

    -- pa


    Mainly due to easy incorporation into UT environment.
    Thursday, November 24, 2011 11:01 PM
  • I have a miniport driver, not protocol - so it's NdisMRegisterMiniportDriver (suppose that doesn't change the answer).
    Of course, you're right!
    I want to understand a bit more regarding the driver instances: suppose I have 2 NICs that I want to manage with the same driver. Will the device manager gui display them as 2 devices?
    Yes.
    Will the OS call the DriverEntry() twice, each time for a different device?
    Only once. MiniportInitialize method will be called twice.  
    How can I know the target device for OS call (e.g. the OS calls the miniport send packets handler - how will I know to which NIC to route the request?) Sorry for all these questions, they probably seem naive :)
    You're right again! :)  On this step you'd probably should find a quiet place and sit there reading WDK docum, googling and running examples in the debugger. (It also helps very much to know complete requirements for the assignment. People rarely write Windows drivers for fun, so I presume there is someone who poses the requirements. )
    Also, why you are so obsessed with operator new. Things can be created without new. Make a factory which internally uses native NdisAllocate.... and returns pointer?
    Mainly due to easy incorporation into UT environment.

    Ah, understood. No real C++ developer can live without UT framework :))  Does your's support Windows kernel? Is it, by chance, CFIX?  If so...  Some time ago I had a short but interesting email exchange with the author of Cfix. We've discussed exactly how "classic" UT frameworks can be adjusted for environments like kernel or embedded - which requires non-trivial steps to get into the context when the test can begin. Everyone has their own pet peeve -  be it mouse mischief or UT in kernel... so I'll stop here until it spins too far into offtopic.

    Regards,

    -- pa 

    Friday, November 25, 2011 7:48 AM
  • How can I know the target device for OS call (e.g. the OS calls the miniport send packets handler - how will I know to which NIC to route the request?) Sorry for all these questions, they probably seem naive :)
    You're right again! :)  On this step you'd probably should find a quiet place and sit there reading WDK docum, googling and running examples in the debugger. (It also helps very much to know complete requirements for the assignment. People rarely write Windows drivers for fun, so I presume there is someone who poses the requirements. )

    The thing is that I actually have read the WDK docs. IMHO, there is one big problem with the documentation. There is an elaborate documentation regarding writing WDM drivers, including DriverEntry, AddDevice and rest of the things a developer should know and, more importantly, the logical connections between everything. Now when it comes to NDIS driver the general picture changes radically, There is no elaborate explanations so there is no big picture. Moreover, that's confusing - many times I wondered whether some piece of info in the WDM driver guidelines is relevant to Ndis drivers. I suppose for everything written in WDM docs there is a parallel adapted functionality (if relevant) in the Ndis. But what is this parallel functionality I can only guess (a good example is setting unload handler for WDM driver compared to Ndis driver).

    This is what I understand so far regarding the questions I've asked: for each NIC the MiniportInitialize will be called and in this handler the driver should allocate new context for the specific NIC and publish it to the OS via NdisMSetMiniportAttributes(). From this point the OS will send this specific driver-provided context as a parameter when it requests anything from the corresponding NIC. Is this correct?

    Ah, understood. No real C++ developer can live without UT framework :))  Does your's support Windows kernel? Is it, by chance, CFIX?  If so...  Some time ago I had a short but interesting email exchange with the author of Cfix. We've discussed exactly how "classic" UT frameworks can be adjusted for environments like kernel or embedded - which requires non-trivial steps to get into the context when the test can begin. Everyone has their own pet peeve -  be it mouse mischief or UT in kernel... so I'll stop here until it spins too far into offtopic.

    Regards,

    -- pa 

    The actual UT framework wasn't defined yet, it's being debated. So I don't have anything specific to say as for now :)
    Friday, November 25, 2011 10:40 AM
  • ice2dude wrote:
    >
    >Can you please comment on my second question?
     
    Do you mean, whether you can use the standard "operator new" in the Win 8
    preview WDK?  I haven't fetched the preview, so I can't answer the question
    unequivocally, but I would be surprised if it were allowed.  The problem is
    that kernel allocations need to specify more information than just the
    size.  You need to tell it whether to allocate paged or non-paged, and you
    need to provide a tag.  The standard "operator new" depends on standard
    library routines that aren't present in the kernel.
    --
    Tim Roberts, timr@probo.com
    Providenza & Boekelheide, Inc.
     

    Tim Roberts, VC++ MVP Providenza & Boekelheide, Inc.
    Saturday, November 26, 2011 7:08 PM
  • IMHO, there is one big problem with the documentation. There is an elaborate documentation regarding writing WDM drivers, including DriverEntry, AddDevice and rest of the things a developer should know and, more importantly, the logical connections between everything. Now when it comes to NDIS driver the general picture changes radically, There is no elaborate explanations so there is no big picture. Moreover, that's confusing - many times I wondered whether some piece of info in the WDM driver guidelines is relevant to Ndis drivers. I suppose for everything written in WDM docs there is a parallel adapted functionality (if relevant) in the Ndis. But what is this parallel functionality I can only guess (a good example is setting unload handler for WDM driver compared to Ndis driver).

    Well this is a good [beginner's ;-)] question, IMHO it deserves a reply for someone who knows better.

    I'd only say briefly that NDIS, during its long history, targeted several kernel architectures such as WinCE and others that we maybe even don't know. Most of instances when you cannot clearly map NDIS concept to WDM are because NDIS has been deliberately designed for portability.

    Regards,

    - pa


    • Edited by Pavel A Sunday, November 27, 2011 12:05 PM
    Sunday, November 27, 2011 11:59 AM
  • Hi again,

    Regarding the handler to be passed to the NdisAllocateMemoryWithTagPriority: if I need to allocate memory in DriverEntry, before actually acquiring the handle from NdisMRegisterMiniportDriver - how can I do such thing?

    And another question regarding the handle: in the MiniportUnload implementation I need to access the handle to pass it to the NdisMDeregisterMiniportDriver. Is there a way to get it from the DriverObject argument?

    Thanks in advance


    • Edited by ice2dude Sunday, January 29, 2012 10:43 AM
    Sunday, January 29, 2012 10:35 AM
  • ice2dude wrote:
    >
    >Yes, I'm aware of these. Just wondered about the new operator - it wasn't
    >previously banned from kernel ...
     
    And, by the way, the WDK include file <kcom.h> includes definitions for
    operator new and delete that solve the problem nicely.
    --
    Tim Roberts, timr@probo.com
    Providenza & Boekelheide, Inc.
     

    Tim Roberts, VC++ MVP Providenza & Boekelheide, Inc.

    Reviving this old thread... Is there a particular reason for having operators new and delete in <kcom.h> but not having new[] and delete[]?

    Thanks again

    Tuesday, November 06, 2012 4:13 PM
  • nothing in particular.  new[] and delete[] are kind of special in that you need to understand how many elements there are to run the constronstructor/destructor on each object.

    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Tuesday, November 06, 2012 5:24 PM