none
Windows 10 Update 1703 problem with wpd and mtp RRS feed

  • Question

  • We have a problem with Windows 10 Update 1703. Our app communicates with the WPD device through MTP. The app worked great, but after Windows update it is not able to write anything to the device. The app is written in C# .NET 4.0 and the code throws an exception: System.AccessViolationException.

    It was tested that this error occurs only on PC with update 1703. Is there anything we can do to make writing to WPD working again?

    Here is the code:

    IPortableDeviceValues values =
    
                GetRequiredPropertiesForContentType(fileName, parentObjectId);
    
            IStream tempStream;
            uint optimalTransferSizeBytes = 0;
    
            content.CreateObjectWithPropertiesAndData(
                values,
                out tempStream,
                ref optimalTransferSizeBytes,
                null);
    
            System.Runtime.InteropServices.ComTypes.IStream targetStream =
                (System.Runtime.InteropServices.ComTypes.IStream)tempStream;
    
            try
            {
                using (var sourceStream =
                       new FileStream(fileName, FileMode.Open, FileAccess.Read))
                {
                    var buffer = new byte[optimalTransferSizeBytes];
                    int bytesRead;
                    do
                    {
                        bytesRead = sourceStream.Read(
                            buffer, 0, (int)optimalTransferSizeBytes);
                        IntPtr pcbWritten = IntPtr.Zero;
                        if (bytesRead < (int)optimalTransferSizeBytes)
                        {
                            targetStream.Write(buffer, bytesRead, pcbWritten);
                        }
                        else
                        {
                            targetStream.Write(buffer, (int)optimalTransferSizeBytes, pcbWritten);
                        }
    
                    } while (bytesRead > 0);
                }
                targetStream.Commit(0);
            }
            finally
            {
                System.Runtime.InteropServices.Marshal.ReleaseComObject(tempStream);
            }
    The System.AccessViolationException exception is on calling targetStream.Write method.
    Friday, June 9, 2017 8:17 AM

Answers

  • Hi

    If you just want the answer, skip to the text in bold!

    Investigating further, the issue is in the low level native API: ISequentialStream::Write (which IStream Inherits) The MSDN page for this is: https://msdn.microsoft.com/en-us/library/windows/desktop/aa380014%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396

    Note the text for the argument: pcbWritten [out] reads
    'A pointer to a ULONG variable where this method writes the actual number of bytes written to the stream object. The caller can set this pointer to NULL, in which case this method does not provide the actual number of bytes written.'

    MS introduced a bug into this API (as called in PortableDeviceApi.dll) --> it is no longer checking whether or not pcbWritten is NULL before attempting to read/write from this variable - rather, it ALWAYS attempts to write the No Of Bytes written into this variable - if the API is called with this variable set to NULL then it BARFS. I have proven this is the case in pure native code by changing the way the API is called:

    Works:

    DWORD bytesWritten   

    IStream->Write(objectData, bytesRead, &bytesWritten)))

       

    Fails:

     IStream->Write(objectData, bytesRead, NULL))) <-- Note the NULL
       

    Up in .Net, IStream.Write is marshalled thus:

        void Write(byte[] pv,int cb,IntPtr pcbWritten)

    and the demo code (from where we all likely got our implementations!) was:

    IntPtr pcbWritten = IntPtr.Zero  //<--Eg NULL     

    IStream.Write(buffer, bytesRead, pcbWritten);

       

    Proposed Solution - Change code to:

    IntPtr pcbWritten = Marshal.AllocHGlobal(4);

    IStream.Write(buffer, bytesRead, pcbWritten);

    Marshal.FreeHGlobal(pcbWritten);


    This works around the issue - Hopefully MS will fix it to avoid us all having to re-distribute our products! The entire code is contained in PortableDeviceApi.dll (also including the stream stuff), which would explain why the entire world is not moaning about this issue and why it was not found during test. NB: For multi block copies in while loop, I suspect the alloc/free can be done outside the while without issue. Should also be safe if MS does fix the bug.

    Credit: Alistair Brown (in our office) for fishing about, and allocating what should not need to be allocated and thus finding the issue.

    Nigel

    • Edited by veletron Tuesday, June 13, 2017 4:53 PM
    • Proposed as answer by veletron Tuesday, June 13, 2017 4:53 PM
    • Marked as answer by krizema Monday, June 19, 2017 8:20 AM
    Tuesday, June 13, 2017 4:51 PM

All replies

  • Also we are facing same issue, even it is crashing my application.


    Friday, June 9, 2017 12:07 PM
  • Hi krizema,

    Thank you for posting here.

    According to the error message we know that an AccessViolationException exception can occur only in unsafe managed code or when verifiable managed code interacts with unmanaged code:

    You can identify and correct the cause of the AccessViolationException exception as follows:

    Make sure that the memory that you are attempting to access has been allocated.

    An AccessViolationException exception is always thrown by an attempt to access protected memory -- that is, to access memory that is not allocated or that is not owned by a process.

    Make sure that the memory that you are attempting to access has not been corrupted.

             If several read or write operations have occurred through bad pointers, memory may be     corrupted. This typically occurs when reading or writing to addresses outside of a predefined buffer.

    Best Regards,

    Hart


    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, June 12, 2017 5:50 AM
  • Hi Hart,

    thank you for your reply. I have the same code as before Windows update and it worked properly. The memory is not corrupted, because I am able to write into the memory in other way, but I also need to write into it through MTP. I am also able to verify, that the problem occurs with more memories so I don't believe that it is a hardware problem. I also found that more programmers have this issue, but I think that we all have the same code.

    targetStream property is not null and everything looks good to me. I am also able to read from the memory so the problem is only with writing.

    Is there anything else I can check?

    Monday, June 12, 2017 7:09 AM
  • Hi,

    According to your description, it seems to be issue on windows 10(1703), i suggest that you can post the issue on connect website that will handle it.

    >>" I am able to write into the memory in other way, but I also need to write into it through MTP.  "

    You can access the memory in other way, when using MTP access the memory, you will get error, so please mark sure that the Hardware is good and usage is correct.

    Best Regards,

    Hart


    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, June 12, 2017 8:35 AM
  • Hi

    I would like to advise that I have this exact same issue writing data to a FileStream on a WPD device.

    The same code worked just fine with the previous windows build 1607 (Millennium  Update) while it fails on build 1703 (Creators Edition).

    I can confirm that if I roll the windows edition back to build 1607, then the exact same executable works just fine!

    Please also note that there are other threads reporting this same issue, for instance: https://stackoverflow.com/questions/43980646/system-accessviolationexception-when-copying-data-to-portable-device-after-insta

    This is definitely a 'feature' introduced into build 1703.

    I am using C# for WPD access, my code is as follows:

      public PortableDeviceFile DoCopyFileToRemote(String fullPathToLocalFile, PortableDeviceObject parent, String remoteFileName, WpdObjectInstance.ReportProgress reportProgress)
            {
                IPortableDeviceContent content;
                _device.Content(out content);
                uint totalBytesWritten = 0;
    
                var iportableDeviceValues = GetRequiredPropertiesForNewFile(fullPathToLocalFile, parent.ObjectId, remoteFileName);
    
                PortableDeviceApiLib.IStream tempStream;
                uint optimalTransferSizeBytes = 0;
                content.CreateObjectWithPropertiesAndData(iportableDeviceValues,out tempStream,ref optimalTransferSizeBytes,null);
                
                optimalTransferSizeBytes = IN_OUT_COPY_BUFFER_SIZE;  //Override default so we can provide meaningful progress feedback (default is 256Kbytes)
                var targetStream = (System.Runtime.InteropServices.ComTypes.IStream)tempStream;
                try
                {
                    using (var sourceStream = new FileStream(fullPathToLocalFile, FileMode.Open, FileAccess.Read))
                    {
                        var progressSize = 100.0/((double)sourceStream.Length/optimalTransferSizeBytes);
                        double progress = 0;
                        var buffer = new byte[optimalTransferSizeBytes];
                        int bytesRead;
                        do
                        {
                            bytesRead = sourceStream.Read(buffer, 0, (int) optimalTransferSizeBytes);
                            IntPtr pcbWritten = IntPtr.Zero;
                            targetStream.Write(buffer, bytesRead, pcbWritten);  // <-- *** FAILURE OCCURS HERE ***
                            totalBytesWritten += (uint) bytesRead;
                            progress += progressSize;
                            if (reportProgress != null) reportProgress((int)progress, remoteFileName);
                        } while (bytesRead > 0);
                        if (sourceStream.Length != totalBytesWritten)
                            return null;
    
                        if (reportProgress != null) reportProgress(100, remoteFileName);
                    }
                    targetStream.Commit(0);
                }
                finally
                {
                    
                    Marshal.ReleaseComObject(tempStream);
                }
    
                //todo - find way to get ID - currently, we find this out via a full file existance check after creation, but it would
                //be better if the system would tell us the ObjectId after creation - it does for new folders!!!
                var portableDeviceFile = new PortableDeviceFile("0", remoteFileName, parent.ObjectId, totalBytesWritten);
                return portableDeviceFile;
            }

    The point where the code fails with an exception is highlighted thus:  '<--Code Fails Here'

    Exception is: 

    An unhandled exception of type 'System.AccessViolationException' occurred in Spacelabs.Shared.Devices.SpacelabsWpdInterfaceWrapper.dll

    Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

    --------------------------

    NB: The memory is NOT corrupt! It can be written to and read from without error by other means. Also, note that an EMPTY file can be created on the WPD device where the above highlighted line is commented out.

    I will endeavour to reproduce using 100% C++ - eliminate the interop from the equation.

    Kind Regards

    Nigel

    Monday, June 12, 2017 4:13 PM
  • PS - I am unable to use the connect website to report this issue - it is not accepting bugs/feedback for windows 10 - least I cannot see windows listed?

    Nigel

    Monday, June 12, 2017 4:17 PM
  • Hi Nigel,

    I was not able to find Windows 10 bugs either, so I reported this problem to the .NET because I am not sure if this problem is in Windows or in framework. Maybe Windows programmers just repaired some bad behaviour and nobody counts with that in framework? It is strange, but it is not impossible...

    One way or the other I hope that repair comes soon, because I am really not happy to explain our customers that the problem is somewhere in Microsoft's code and not in ours.

    Tuesday, June 13, 2017 6:50 AM
  • See the link below for a repeater for the issue - look at the readme file - you will need to call for your specific WPD device.

    Repeater for failure - see readme file

    Note the SO thread on the same issue here: https://stackoverflow.com/questions/43980646/system-accessviolationexception-when-copying-data-to-portable-device-after-insta

    The Microsoft Connect bug report for the issue may be found here: https://connect.microsoft.com/VisualStudio/Feedback/Details/3135829 If you have the same issue please visit and click the repro link to indicate as such. 

    Nigel


    • Edited by veletron Tuesday, June 13, 2017 1:24 PM make links active
    • Proposed as answer by mdebus Tuesday, June 13, 2017 2:05 PM
    • Unproposed as answer by mdebus Tuesday, June 13, 2017 2:05 PM
    Tuesday, June 13, 2017 1:00 PM
  • Hi

    If you just want the answer, skip to the text in bold!

    Investigating further, the issue is in the low level native API: ISequentialStream::Write (which IStream Inherits) The MSDN page for this is: https://msdn.microsoft.com/en-us/library/windows/desktop/aa380014%28v=vs.85%29.aspx?f=255&MSPPError=-2147217396

    Note the text for the argument: pcbWritten [out] reads
    'A pointer to a ULONG variable where this method writes the actual number of bytes written to the stream object. The caller can set this pointer to NULL, in which case this method does not provide the actual number of bytes written.'

    MS introduced a bug into this API (as called in PortableDeviceApi.dll) --> it is no longer checking whether or not pcbWritten is NULL before attempting to read/write from this variable - rather, it ALWAYS attempts to write the No Of Bytes written into this variable - if the API is called with this variable set to NULL then it BARFS. I have proven this is the case in pure native code by changing the way the API is called:

    Works:

    DWORD bytesWritten   

    IStream->Write(objectData, bytesRead, &bytesWritten)))

       

    Fails:

     IStream->Write(objectData, bytesRead, NULL))) <-- Note the NULL
       

    Up in .Net, IStream.Write is marshalled thus:

        void Write(byte[] pv,int cb,IntPtr pcbWritten)

    and the demo code (from where we all likely got our implementations!) was:

    IntPtr pcbWritten = IntPtr.Zero  //<--Eg NULL     

    IStream.Write(buffer, bytesRead, pcbWritten);

       

    Proposed Solution - Change code to:

    IntPtr pcbWritten = Marshal.AllocHGlobal(4);

    IStream.Write(buffer, bytesRead, pcbWritten);

    Marshal.FreeHGlobal(pcbWritten);


    This works around the issue - Hopefully MS will fix it to avoid us all having to re-distribute our products! The entire code is contained in PortableDeviceApi.dll (also including the stream stuff), which would explain why the entire world is not moaning about this issue and why it was not found during test. NB: For multi block copies in while loop, I suspect the alloc/free can be done outside the while without issue. Should also be safe if MS does fix the bug.

    Credit: Alistair Brown (in our office) for fishing about, and allocating what should not need to be allocated and thus finding the issue.

    Nigel

    • Edited by veletron Tuesday, June 13, 2017 4:53 PM
    • Proposed as answer by veletron Tuesday, June 13, 2017 4:53 PM
    • Marked as answer by krizema Monday, June 19, 2017 8:20 AM
    Tuesday, June 13, 2017 4:51 PM
  • To add a little more detail to this from the crash dumps we have seen there appears to be an error introduced with a call to TransferInfo::StopTransfer() present in function CPortableDeviceCreateObjectStream::Write() in PortableDeviceAPI.dll.  StopTransfer() appears to be passed the equivalent of *pcbWritten, regardless of whether this is NULL or not.  This deferences a NULL pointer and causes a crash when encountered.

    Note that this behaviour affects both 32 and 64 bit versions of the DLL and is new behaviour introduced in 1703 that was not present in build 1607 (anniversary edition).


    Wednesday, June 14, 2017 5:03 PM
  • Windows 10 has a new (I think) mechanism built in for bug reporting and upvoting:

    1) Start the 'Feedback Hub' App
    2) Search for '1703 windows portable device'  (My bug report for this issue)
    3) Pick the item 'Windows 10 Build 1703 (Creators Update) bug in Windows Portable Device IStream:Write code - writes to device fail with System.AccessViolationException'
    4) Upvote and comment!
    Thursday, June 15, 2017 2:57 PM
  • Thank you both veletron and Alistair. I owe you a beer.

    @veletron I was not able to find your report in that "amazing app". It shows me some bug reports, but I really doubt that it is a full list of reported bugs.

    Monday, June 19, 2017 9:01 AM
  • Here's the link to the bug in the 'Feedback Hub' 

    https://aka.ms/Eiczjq


    • Edited by veletron Monday, June 19, 2017 9:46 AM
    Monday, June 19, 2017 9:45 AM
  • Great, thank you. Upvoted. I don't want to know, how and why they filter reports for me...
    Monday, June 19, 2017 10:04 AM
  • Thanks for your help !

    If y can help you, I corrected two functions (TransferContentToDevice and DownloadFile) :

    public void TransferContentToDevice(string fileName, string parentObjectId)
            {
                IPortableDeviceContent content;
                this._device.Content(out content);

                IPortableDeviceValues values = GetRequiredPropertiesForContentType(fileName, parentObjectId);

                PortableDeviceApiLib.IStream tempStream;
                uint optimalTransferSizeBytes = 0;
                content.CreateObjectWithPropertiesAndData(values, out tempStream, ref optimalTransferSizeBytes, null);

                System.Runtime.InteropServices.ComTypes.IStream targetStream = (System.Runtime.InteropServices.ComTypes.IStream)tempStream;
                try
                {
                    using (var sourceStream = new FileStream(fileName, FileMode.Open, FileAccess.Read))
                    {
                        var buffer = new byte[optimalTransferSizeBytes];
                        int bytesRead;
                        do
                        {
                            bytesRead = sourceStream.Read(buffer, 0, (int)optimalTransferSizeBytes);
                            IntPtr pcbWritten = Marshal.AllocHGlobal(4);
                            targetStream.Write(buffer, bytesRead, pcbWritten);
                            Marshal.FreeHGlobal(pcbWritten);
                        } while (bytesRead > 0);
                    }
                    targetStream.Commit(0);
                }
                finally
                {
                    Marshal.ReleaseComObject(tempStream);
                }
            }

    public void DownloadFile(PortableDeviceFile file, string saveToPath)
            {
                IPortableDeviceContent content;
                this._device.Content(out content);

                IPortableDeviceResources resources;
                content.Transfer(out resources);

                PortableDeviceApiLib.IStream wpdStream;
                uint optimalTransferSize = 0;
                
                var property = new _tagpropertykey();
                property.fmtid = new Guid(0xE81E79BE, 0x34F0, 0x41BF, 0xB5, 0x3F, 0xF1, 0xA0, 0x6A, 0xE8, 0x78, 0x42);
                property.pid = 0;
                
                resources.GetStream(file.Id, ref property, 0, ref optimalTransferSize, out wpdStream);

                System.Runtime.InteropServices.ComTypes.IStream sourceStream = (System.Runtime.InteropServices.ComTypes.IStream)wpdStream;
                try
                {
                    var filename = file.Name;

                    using (StreamWriter targetStream = new StreamWriter(Path.Combine(saveToPath, filename), false))
                    {
                        var buffer = new byte[1024];
                        int bytesRead;
                        do
                        {
                            IntPtr ReadBuffer = Marshal.AllocHGlobal(4);
                            sourceStream.Read(buffer, buffer.Length, ReadBuffer);
                            bytesRead = Marshal.ReadInt32(ReadBuffer);
                            Marshal.FreeHGlobal(ReadBuffer);
                            if (bytesRead > 0)
                            {
                                targetStream.BaseStream.Write(buffer, 0, bytesRead);
                            }
                        }
                        while (bytesRead > 0);
                    }
                }
                finally
                {
                    Marshal.ReleaseComObject(wpdStream);
                }
            }

    Sorry for my english, I'm french ;-)

    Thursday, August 31, 2017 10:19 AM