none
Mapping memory on 64b OS from a 32b application and DLL RRS feed

  • Question

  • Hi,

    I have developed a PCIe driver and a dll with an application that access a DDR memory.

    The application can do a mmap on a DDR memory area (allocation implemented in the driver with an IOCTL and the function MmMapIoSpace() ). By passing the physical address to the function i get the virtual address that i can use to read/write the memory area.

    This process works fine when i use it on a 32b OS with a 32b dll/application. The process works fine when I use it on a 64b OS with a 64b dll/application.

    However, when i use the 32b dll/application on a 64b OS, the mmap crashes. The system crashes (BSOD) on the function MmMapIoSpace().

    I know that the OS emulates the application with the layer WOW64 but i don't know if there are some restrictions about mapping (mmap).

    If someone knows if the process should work or another information about mmap functionality in mixed mode, I'm very interested to learn it.

    Thank you.


    • Edited by xpxp1984 Thursday, August 9, 2018 12:56 PM
    Thursday, August 9, 2018 11:44 AM

Answers

  • Hi all,

    My issue is quite resolved. The problem is that my structure with all my parameters coming from my 32b library is not well aligned when it is recovered in the driver. By the way some parameters are unaligned. I need to use the #pragma pack to correct the issue.

    Thank you for the help !

    • Marked as answer by xpxp1984 Tuesday, August 14, 2018 8:51 AM
    Tuesday, August 14, 2018 8:50 AM

All replies

  • - Doctor, when I do this, it pains!
    - Do not do this!

    Thursday, August 9, 2018 12:52 PM
  • Hum.. So is there a better method ? Thanks.
    Thursday, August 9, 2018 12:54 PM
  • Post your mapping code and the output of !analyze -v

     -Brian


    Azius Developer Training www.azius.com Windows device driver, internals, security, & forensics training and consulting. Blog at www.azius.com/blog

    Thursday, August 9, 2018 6:26 PM
    Moderator
  • I'm confused by your description.  MmMapIoSpace takes a device physical address and returns a kernel virtual address.  Usually, you do that exactly once, in the start device handler, to map your device's bar into kernel space.

    MmMapIoSpace doesn't play in user space.  So, exactly what are you doing in your ioctl?  Did you mean MmMapLockedPagesSpecifyCache?

    Mapping device memory directly into user mode is a security hole, because user-mode processes aren't secure.  You might consider if it would be smarter to architect things differently, so that the kernel driver has sole access to the hardware.


    Tim Roberts | Driver MVP Emeritus | Providenza & Boekelheide, Inc.

    Friday, August 10, 2018 4:41 AM
  • Hi,

    Thank you for the answer.

    You are right, after calling MmMapIoSpace() I use MmMapLockedPagesSpecifyCache() to get the virtual address. However it really seems that my system crashes when the MmMapIoSpace() function is called.

    I know that the methodology is not the good under the Windows philosophy. This project is mainly inspired by a Linux version that we have. It is a specific application. This project allows to control PCIE-VME board. I precise that this code works well in native 32b or native 64b. Below the structure of the code :

    User application
    ----------------

    .
    .
    .
    // Fill mapping structure MEMORY
    memset(&map_loc_win, 0, sizeof(map_loc_win));
    map_loc_win.req.rem_addr   = 0;      // Address in SHM1
    map_loc_win.req.loc_addr   = 0; // Local address
    map_loc_win.req.size       = 0x100000; // Mapping size
    map_loc_win.req.mode.sg_id = MAP_ID_MAS_PCIE_MEM; // Memory space
    map_loc_win.req.mode.space = MAP_SPACE_SHM; // SHM #1
    map_loc_win.req.mode.flags = 0; // No flag

    // Allocate a mapping window by acquiring the local physical address of the area
    retval = alt_map_alloc(&map_loc_win);

    // Map device memory MEM
    uaddr = MmapAltDrv(tc->fd, map_loc_win.req.loc_addr + offset, map_loc_win.req.size);
    .
    .
    .


    DLL library
    -----------

    void * MmapAltDrv(int fd, unsigned _int64 p_addr, int size) {
    ulong BytesReturned;
    struct alt_mmap_req *mr;
    int idx;
    void* malloc_buf = TlsGetMallocBuf(MALLOC_SIZE);

    if (fd < 0) {
    return (NULL);
    }
    if (p_addr < 0x80000000)
    return NULL;
    if (alt_mdl_set >= max_mdl - 1) {
    return (NULL);
    }
    mr = (struct alt_mmap_req *) malloc_buf;
    mr->p_addr = p_addr;
    mr->size   = size;
    mr->Mdl    = NULL;
    if (!DeviceIoControl(hDriver, IOCTL_ALTDRV_MMAP, (void *)mr, sizeof(struct alt_mmap_req), (void *)mr, sizeof(struct alt_mmap_req), (LPDWORD)& BytesReturned, NULL))
    {
    return NULL;
    }
    if (BytesReturned > 0) {
    alt_mdl_set++;
    for (idx = 0; idx < max_mdl; idx++)     // find next available array element
    {      
    if (alt_mdl[idx].v_addr == NULL) {
    break;
    }
    }
    alt_mdl[idx].v_addr = mr->v_addr;
    alt_mdl[idx].size   = mr->size;
    alt_mdl[idx].Mdl    = mr->Mdl;
    return (mr->v_addr);
    }
    return (NULL);
    }

    Driver
    ------

    VOID AltheaWDFEvtIoDeviceControl(_In_ WDFQUEUE Queue, _In_ WDFREQUEST Request, _In_ size_t OutputBufferLength, _In_ size_t InputBufferLength, _In_ ULONG IoControlCode){
    WDFDEVICE hDevice = WdfIoQueueGetDevice(Queue);
    PDEVICE_CONTEXT pdx = DeviceGetContext(hDevice);

    NTSTATUS ntStatus = STATUS_SUCCESS;
    PVOID inputIOBuffer;
    PVOID outputIOBuffer;
    SIZE_T xferLen = 0;
    struct tagPhysStruct PhysStruct;
    struct tagPortStruct PortStruct;

    ntStatus = WdfRequestRetrieveInputBuffer(Request, 0, &inputIOBuffer, NULL);
    if (!NT_SUCCESS(ntStatus)) {
    WdfRequestComplete(Request, ntStatus);
    return;
    }
    ntStatus = WdfRequestRetrieveOutputBuffer(Request, 0, &outputIOBuffer, NULL);
    if (!NT_SUCCESS(ntStatus)) {
    WdfRequestComplete(Request, ntStatus);
    return;
    }

    ntStatus = STATUS_SUCCESS;
    switch (IoControlCode) {
    .
    .
    .
    case IOCTL_ALTDRV_MMAP:
    xferLen = alt_mmap(pdx, outputIOBuffer, (int)InputBufferLength, (int)OutputBufferLength);
    if ((int)xferLen < 0) {
    ntStatus = STATUS_INVALID_PARAMETER;
    }
    break;
    .
    .
    .

    if (ntStatus != STATUS_PENDING) {
    if (NT_SUCCESS(ntStatus))
    WdfRequestCompleteWithInformation(Request, ntStatus, xferLen);
    else
    WdfRequestComplete(Request, ntStatus);
    }

        return;
    }

    Driver
    ------

    int alt_mmap(PDEVICE_CONTEXT pdx, void *iobuf, int in_len, int out_len) {
    UNREFERENCED_PARAMETER(in_len);
    UNREFERENCED_PARAMETER(pdx);
    unsigned _int64 p_addr;
    struct alt_mmap_req *mr;
    PMDL Mdl;
    int size;

    mr   = (struct alt_mmap_req *) iobuf;
    size = mr->size;
    mr->v_addr = NULL;
    if (mr->p_addr >= 0x80000000){
    p_addr     = (unsigned _int64)mr->p_addr;
    mr->k_addr = (unsigned _int64)ioremap(p_addr, size);
    }
    else{
    p_addr = (unsigned _int64)mr->p_addr;
    mr->size |= 0x80000000;
    }
    Mdl = IoAllocateMdl((PVOID)mr->k_addr, size, FALSE, FALSE, NULL);
    mr->Mdl = (void *) Mdl;
    if (!Mdl) {
    return (-1);
    }
    MmBuildMdlForNonPagedPool(Mdl);
    __try {
    mr->v_addr = (PVOID)(((ulong)PAGE_ALIGN(MmMapLockedPagesSpecifyCache(Mdl, UserMode, MmNonCached, NULL, FALSE, NormalPagePriority))) + MmGetMdlByteOffset(Mdl));
    }
    __except (EXCEPTION_EXECUTE_HANDLER) {
    TraceEvents(TRACE_LEVEL_ERROR, TRACE_QUEUE, "%!FUNC! Failed MmMapLockedPagesSpecifyCache");
    return (-1);
    }
    return (out_len);
    }

    Driver
    ------

    void * ioremap(unsigned _int64 addr, int size) {
    PHYSICAL_ADDRESS phys_addr;
    void * v_addr = NULL;

    phys_addr.QuadPart = addr;
    v_addr = MmMapIoSpace(phys_addr, size, MmNonCached);
    return (v_addr);
    }

    Thank you for the help and for your advises.

    Friday, August 10, 2018 7:41 AM
  • The most likely failure here is that you are giving it an invalid address range of physical memory.   You stated that is crashes with a BSOD, get a crash dump and post the !analyze -v here making sure the symbols are correct so we can use it.

    By the way, as Tim warned this is a security hole, and your driver is one of the worst on this I have seen.


    Don Burn Windows Driver Consulting Website: http://www.windrvr.com

    Friday, August 10, 2018 10:52 AM
  • also, there is no guarantee your queue is presenting the request in the context of the calling process.  If you need to guarantee to be in the right context, you need to register an in process callback

    d -- This posting is provided "AS IS" with no warranties, and confers no rights.

    Friday, August 10, 2018 2:01 PM
  • By the way, as Tim warned this is a security hole

    Is this reasoning still actual? These days there are integrity levels... and user mode drivers, are they not in usermode after all?

    I have very hard times maintaining this POV before Linux-minded folks. Who use mmap routinely everywhere, as you know.

    Is there a better, more modern explanation? For example, common injection of various stuff into usermode processes, which can touch the i/o memory by accident? Again, can proper integrity labeling protect against this?

    Regards,

    -- pa



    • Edited by Pavel A Friday, August 10, 2018 2:05 PM
    Friday, August 10, 2018 2:04 PM
  • Hi all,

    Thank you for your answers. I'm aware on the fact that the methodology is not good in "Windows environment". However, i need to live with it for the moment.

    Below, the dump of the BSOD. It seems that it is well the MmapIoSpace() function that causes the BSOD.

    Mainly i would like to know if this methodology could be work in a mixed environment or not. In the future i will redevelop this project in the good manner.

    Thanks.

    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    Use !analyze -v to get detailed debugging information.

    BugCheck 50, {fffff6fc40178000, 1, fffff8000329076a, 9}

    *** WARNING: Unable to verify timestamp for AWDF.sys
    *** ERROR: Module load completed but symbols could not be loaded for AWDF.sys

    Could not read faulting driver name
    Probably caused by : AWDF.sys ( AWDF+18bb )

    Followup:     MachineOwner
    ---------

    1: kd> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Bugcheck Analysis                                    *
    *                                                                             *
    *******************************************************************************

    PAGE_FAULT_IN_NONPAGED_AREA (50)
    Invalid system memory was referenced.  This cannot be protected by try-except.
    Typically the address is just plain bad or it is pointing at freed memory.
    Arguments:
    Arg1: fffff6fc40178000, memory referenced.
    Arg2: 0000000000000001, value 0 = read operation, 1 = write operation.
    Arg3: fffff8000329076a, If non-zero, the instruction address which referenced the bad memory
    address.
    Arg4: 0000000000000009, (reserved)

    Debugging Details:
    ------------------


    Could not read faulting driver name

    DUMP_CLASS: 1

    DUMP_QUALIFIER: 400

    BUILD_VERSION_STRING:  7601.24168.amd64fre.win7sp1_ldr.180608-0600

    SYSTEM_MANUFACTURER:  Supermicro

    SYSTEM_PRODUCT_NAME:  Super Server

    SYSTEM_SKU:  To be filled by O.E.M.

    SYSTEM_VERSION:  0123456789

    BIOS_VENDOR:  American Megatrends Inc.

    BIOS_VERSION:  1.0b

    BIOS_DATE:  05/05/2016

    BASEBOARD_MANUFACTURER:  Supermicro

    BASEBOARD_PRODUCT:  X11SSH-F

    BASEBOARD_VERSION:  1.01

    DUMP_TYPE:  2

    BUGCHECK_P1: fffff6fc40178000

    BUGCHECK_P2: 1

    BUGCHECK_P3: fffff8000329076a

    BUGCHECK_P4: 9

    WRITE_ADDRESS: GetPointerFromAddress: unable to read from fffff800034ae100
    Unable to get MmSystemRangeStart
    GetUlongPtrFromAddress: unable to read from fffff800034ae2e8
    GetUlongPtrFromAddress: unable to read from fffff800034ae4a0
    GetPointerFromAddress: unable to read from fffff800034ae0d8
     fffff6fc40178000 

    FAULTING_IP: 
    nt!MmMapIoSpace+d0a
    fffff800`0329076a 49895d00        mov     qword ptr [r13],rbx

    MM_INTERNAL_CODE:  9

    CPU_COUNT: 4

    CPU_MHZ: cf0

    CPU_VENDOR:  GenuineIntel

    CPU_FAMILY: 6

    CPU_MODEL: 5e

    CPU_STEPPING: 3

    CPU_MICROCODE: 6,5e,3,0 (F,M,S,R)  SIG: 8A'00000000 (cache) 8A'00000000 (init)

    CUSTOMER_CRASH_COUNT:  1

    DEFAULT_BUCKET_ID:  WIN7_DRIVER_FAULT

    BUGCHECK_STR:  0x50

    PROCESS_NAME:  ATST.exe

    CURRENT_IRQL:  0

    ANALYSIS_SESSION_HOST: 

    ANALYSIS_SESSION_TIME:  08-10-2018 14:37:14.0585

    ANALYSIS_VERSION: 10.0.16299.15 amd64fre

    TRAP_FRAME:  fffff88011a07260 -- (.trap 0xfffff88011a07260)
    NOTE: The trap frame does not contain all registers.
    Some register values may be zeroed or incorrect.
    rax=00000000d8034000 rbx=0000000000000000 rcx=fffff6fc40178000
    rdx=fffff80003434700 rsi=0000000000000000 rdi=0000000000000000
    rip=fffff8000329076a rsp=fffff88011a073f0 rbp=fffff6fc40178000
     r8=0000000000000000  r9=fffff88011a07388 r10=00000000000d8034
    r11=fffffa8012673500 r12=0000000000000000 r13=0000000000000000
    r14=0000000000000000 r15=0000000000000000
    iopl=0         nv up ei pl nz na po nc
    nt!MmMapIoSpace+0xd0a:
    fffff800`0329076a 49895d00        mov     qword ptr [r13],rbx ds:00000000`00000000=????????????????
    Resetting default scope

    LAST_CONTROL_TRANSFER:  from fffff80003382bf3 to fffff800032af8a0

    STACK_TEXT:  
    fffff880`11a07108 fffff800`03382bf3 : 00000000`00000050 fffff6fc`40178000 00000000`00000001 fffff880`11a07260 : nt!KeBugCheckEx
    fffff880`11a07110 fffff800`032bba96 : 00000000`00000001 fffff6fc`40178000 00000000`00000000 00000000`d803487b : nt!MmAccessFault+0x1ea3
    fffff880`11a07260 fffff800`0329076a : 00000000`00000000 00000000`00000000 00000000`00000001 fffffa80`00000000 : nt!KiPageFault+0x356
    fffff880`11a073f0 fffff880`0e9958bb : 00000000`d8034000 00000000`00000000 fffffa80`00011b91 00000000`00000000 : nt!MmMapIoSpace+0xd0a
    fffff880`11a07510 00000000`d8034000 : 00000000`00000000 fffffa80`00011b91 00000000`00000000 00000000`0000001c : AWDF+0x18bb
    fffff880`11a07518 00000000`00000000 : fffffa80`00011b91 00000000`00000000 00000000`0000001c 00000000`002220a4 : 0xd8034000


    THREAD_SHA1_HASH_MOD_FUNC:  b3c60d67ec6c912122458a3c33cd2176307d1fe3

    THREAD_SHA1_HASH_MOD_FUNC_OFFSET:  f646a045419e969752231f972075e6c4b97bcb2a

    THREAD_SHA1_HASH_MOD:  d9c496685fe8b907ee407ddfe8f58568239fc7f1

    FOLLOWUP_IP: 
    AWDF+18bb
    fffff880`0e9958bb ??              ???

    SYMBOL_STACK_INDEX:  4

    SYMBOL_NAME:  AWDF+18bb

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: AWDF

    IMAGE_NAME:  AWDF.sys

    DEBUG_FLR_IMAGE_TIMESTAMP:  5b6d4f8f

    STACK_COMMAND:  .thread ; .cxr ; kb

    FAILURE_BUCKET_ID:  X64_0x50_AWDF+18bb

    BUCKET_ID:  X64_0x50_AWDF+18bb

    PRIMARY_PROBLEM_CLASS:  X64_0x50_AWDF+18bb

    TARGET_TIME:  2018-08-10T12:07:37.000Z

    OSBUILD:  7601

    OSSERVICEPACK:  1000

    SERVICEPACK_NUMBER: 0

    OS_REVISION: 0

    SUITE_MASK:  272

    PRODUCT_TYPE:  1

    OSPLATFORM_TYPE:  x64

    OSNAME:  Windows 7

    OSEDITION:  Windows 7 WinNt (Service Pack 1) TerminalServer SingleUserTS

    OS_LOCALE:  

    USER_LCID:  0

    OSBUILD_TIMESTAMP:  2018-06-08 17:34:37

    BUILDDATESTAMP_STR:  180608-0600

    BUILDLAB_STR:  win7sp1_ldr

    BUILDOSVER_STR:  6.1.7601.24168.amd64fre.win7sp1_ldr.180608-0600

    ANALYSIS_SESSION_ELAPSED_TIME:  38e

    ANALYSIS_SOURCE:  KM

    FAILURE_ID_HASH_STRING:  km:x64_0x50_awdf+18bb

    FAILURE_ID_HASH:  {20ea68de-9719-c106-7b66-1312cc878f0a}

    Followup:     MachineOwner
    ---------

    Friday, August 10, 2018 2:11 PM
  • It looks like you are trying to read physical address 0, I don't believe that is a valid physical address.   You need to get the RESOURCE_LIST from

    "\REGISTRY\MACHINE\HARDWARE\RESOURCEMAP\System Resources\Physical Memory" and only attempt to read a valid memory address.  As I said your driver has a security hole, and makes it incredibly easy for any application to crash the computer it is running on.

    To answer another posters question, yes it is a security hole in several ways even with the newest versions of Windows.


    Don Burn Windows Driver Consulting Website: http://www.windrvr.com

    Friday, August 10, 2018 3:11 PM
  • Hi all,

    My issue is quite resolved. The problem is that my structure with all my parameters coming from my 32b library is not well aligned when it is recovered in the driver. By the way some parameters are unaligned. I need to use the #pragma pack to correct the issue.

    Thank you for the help !

    • Marked as answer by xpxp1984 Tuesday, August 14, 2018 8:51 AM
    Tuesday, August 14, 2018 8:50 AM