locked
Parameter type UINT of nBufferSize of ctor CMemFile::CMemFile(BYTE* lpBuffer, UINT nBufferSize, UINT nGrowBytes) and CMemFile::Attach is wrong - Visual Studio Prof. 2015 Update 2 64 bit build RRS feed

  • Question

  • Hello,

    CMemFile supports SIZE_T m_nBufferSize but I can't use CMemFile::Attach or the ctor because the nBufferSize parameter is of type UINT.

    I need Attach with SIZE_T, because I want to attach the buffer returned by MapViewOfFile that has as parameter dwNumberOfBytesToMap of type SIZE_T, see:

      BYTE *pBuffer = (BYTE* ) ::MapViewOfFile( (HANDLE) m_hFile,
        dwDesiredAccess,
        dwFileOffsetHigh, 
        dwFileOffsetLow,
        dwNumberOfBytesToMap);
      
      if (!pBuffer)
      {
        if ( m_bShowMessageBox )
          ::AfxMessageBox( _T("Unable to map view of a file") );
        return FALSE;
      }
      
    CMemFile::Attach(pBuffer, dwNumberOfBytesToMap, 0)

    Thanks for the help,


    Andreas


    Tuesday, July 26, 2016 12:59 PM

Answers

  • Yes, this
        void Attach(BYTE* lpBuffer, SIZE_T nBufferSize, SIZE_T nGrowBytes = 0)
        {
          // Code copied from filemem.cpp CMemFile::Attach
          if (lpBuffer == NULL && nBufferSize != 0)
          {
            AfxThrowInvalidArgException();
          }
    
          ASSERT(m_lpBuffer == NULL);
    
          m_nGrowBytes = nGrowBytes;
          m_nPosition = 0;
          m_nBufferSize = nBufferSize;
          m_nFileSize = nGrowBytes == 0 ? nBufferSize : 0;
          m_lpBuffer = lpBuffer;
          m_bAutoDelete = FALSE;
        }

    is a possible workaround - but it would be nice if the parameter is changed to SIZE_T in all functions / ctor in the future.

    Andreas

    If you believe that my suggestion and the code that I provided is the answer to your question I would appreciate it if you would mark my response as the answer instead of marking your own response that contains a copy of my suggested solution.

    Thursday, July 28, 2016 12:49 PM

All replies

  • Have you tried this?

    CMemFile::Attach(pBuffer, (UINT)dwNumberOfBytesToMap, 0)


    Cheers
    Eddie

    Tuesday, July 26, 2016 1:14 PM
  • SIZE_T is a typedef for either unsigned long or unsigned __int64.  In either case, there should be implicit conversions between SIZE_T and UINT.

    When you compile the code, what diagnostic does the compiler produce?

    Tuesday, July 26, 2016 4:21 PM
  • Hi Andreas Schniertshauer,

    thanks for posting here.

    Seems like the type of dwNumberOfBytesToMap is DWORD. If my understanding is right, your case is actually about how to convert DWORD to UINT. Just like Eddie Lotter suggested, use explicit conversion.

    If not, you want to convert SIZE_T to UINT. You could try ULongPtrToUInt function.

    SIZE_T is the maximum number of bytes to which a pointer can point. Use for a count that must span the full range of a pointer.

    This type is declared in BaseTsd.h as follows:

    typedef ULONG_PTR SIZE_T;

    An unsigned INT. The range is 0 through 4294967295 decimal.

    This type is declared in WinDef.h as follows:

    typedef unsigned int UINT;

    The *_PTR types were added to the Windows API in order to support Win64's 64bit addressing.

    Because 32bit APIs are used to store pointers to things using data types like DWORDS, it was necessary to create new types for 64 bit compatibility that acted like a DWORD in 32bits, but were extended to 64bits when used in a 64bit app.

    For more information about data types, please refer to this link below.

    https://msdn.microsoft.com/en-us/library/windows/desktop/aa383751(v=vs.85).aspx

    Best Regards,

    Sera Yu


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.

    Wednesday, July 27, 2016 1:54 AM
  •           // In x64 build
              auto soui = sizeof(UINT);   // 4
              auto sost = sizeof(SIZE_T); // 8
              ASSERT(sost <= soui);       // SIZE_T -> UINT -> bang - loss of data
    


    Andreas

    Wednesday, July 27, 2016 6:17 AM
  • Data is only lost if the value of the SIZE_T variable exceeds UINT_MAX.  In that case, the loss occurs regardless of how the conversion is performed, implicitly by assignment or explicitly with a cast.
    Wednesday, July 27, 2016 9:07 AM
  • The ULongPtrToUInt conversion function recommended by Sera Yu does error checking and will return an HRESULT error code where conversion causes a truncation error.  At least this method allows your code to detect a potential problem and respond accordingly.

    Wednesday, July 27, 2016 10:34 AM
  • I want to pass values greater than  UINT_MAX

    Andreas

    Wednesday, July 27, 2016 1:25 PM
  • I don't need error checking I want to use the function with values greater than UNIT_MAX, because I want to attach the buffer returned by 
    MapViewOfFile

    Andreas

    Wednesday, July 27, 2016 1:27 PM
  • The CMemFile implementation is in the MFC source file filemem.cpp.  The class variable m_nBufferSize is of type SIZE_T.  Unfortunately, the Attach function is not virtual.  However, you could copy the implementation code out of filemem.cpp into your own class and change it so that your Attach function will accept a SIZE_T parameter instead of the undesired UINT for the buffer size.


    Just for fun, take a look at this --

    class CMyMemFile : public CMemFile
    {
    public:
    	CMyMemFile() {};
    	~CMyMemFile() {};
    	void Attach(BYTE* lpBuffer, SIZE_T nBufferSize, UINT nGrowBytes = 0)
    	{
    		if (lpBuffer == NULL && nBufferSize != 0)
    		{
    			AfxThrowInvalidArgException();
    		}
    
    		ASSERT(m_lpBuffer == NULL);
    
    		m_nGrowBytes = nGrowBytes;
    		m_nPosition = 0;
    		m_nBufferSize = nBufferSize;
    		m_nFileSize = nGrowBytes == 0 ? nBufferSize : 0;
    		m_lpBuffer = lpBuffer;
    		m_bAutoDelete = FALSE;
    
    	};
    };

    • Proposed as answer by Eddie Lotter Wednesday, July 27, 2016 2:15 PM
    • Edited by RLWA32 Wednesday, July 27, 2016 2:26 PM
    Wednesday, July 27, 2016 2:03 PM
  • On 7/27/2016 9:25 AM, Andreas Schniertshauer wrote:

    I want to pass values greater than  UINT_MAX

    In this case, you cannot use CMemFile. It does not support working with a block of memory larger than 4GB. Or rather, you can use it, but you'd have to write your data in portions, each smaller than 4GB.

    Wednesday, July 27, 2016 7:16 PM
  • Yes, this
        void Attach(BYTE* lpBuffer, SIZE_T nBufferSize, SIZE_T nGrowBytes = 0)
        {
          // Code copied from filemem.cpp CMemFile::Attach
          if (lpBuffer == NULL && nBufferSize != 0)
          {
            AfxThrowInvalidArgException();
          }
    
          ASSERT(m_lpBuffer == NULL);
    
          m_nGrowBytes = nGrowBytes;
          m_nPosition = 0;
          m_nBufferSize = nBufferSize;
          m_nFileSize = nGrowBytes == 0 ? nBufferSize : 0;
          m_lpBuffer = lpBuffer;
          m_bAutoDelete = FALSE;
        }

    is a possible workaround - but it would be nice if the parameter is changed to SIZE_T in all functions / ctor in the future.

    Andreas



    Wednesday, July 27, 2016 9:25 PM
  • In this case, you cannot use CMemFile. It does not support working with a block of memory larger than 4GB. Or rather, you can use it, but you'd have to write your data in portions, each smaller than 4GB.
    Is the 4GB limitation referred to above related to 32 bit memory allocation limits as opposed to the design and implementation of CMemFile? 
    Wednesday, July 27, 2016 10:56 PM
  • On 7/27/2016 6:56 PM, RLWA32 wrote:

    In this case, you cannot use CMemFile. It does not support working with a block of memory larger than 4GB. Or rather, you can use it, but you'd have to write your data in portions, each smaller than 4GB.

    Is the 4GB limitation referred to above related to 32 bit memory allocation limits as opposed to the design and implementation of CMemFile?

    Just the design of CMemFile (assuming you are building a 64-bit binary, of course). It looks more like a bug, really; or perhaps backward compatibility constraints. The class' private members are all SIZE_T (64 bit on 64-bit system), but the public interface was not updated accordingly, and uses UINT (always 32 bit). So the limitation appears to be mostly artificial.

    Wednesday, July 27, 2016 11:18 PM
  • On 7/27/2016 6:56 PM, RLWA32 wrote:

    In this case, you cannot use CMemFile. It does not support working with a block of memory larger than 4GB. Or rather, you can use it, but you'd have to write your data in portions, each smaller than 4GB.

    Is the 4GB limitation referred to above related to 32 bit memory allocation limits as opposed to the design and implementation of CMemFile?

    Just the design of CMemFile (assuming you are building a 64-bit binary, of course). It looks more like a bug, really; or perhaps backward compatibility constraints. The class' private members are all SIZE_T (64 bit on 64-bit system), but the public interface was not updated accordingly, and uses UINT (always 32 bit). So the limitation appears to be mostly artificial.


    So in a 64 bit world the CMemFile class can handle a 4GB+ memory buffer but would be forced to manipulate it in an awkward fashion through  the UINT sized public interface for read, write, seek, etc.
    Wednesday, July 27, 2016 11:29 PM
  • On 7/27/2016 7:29 PM, RLWA32 wrote:

    So in a 64 bit world the CMemFile class can handle a 4GB+ memory buffer but would be forced to manipulate it in an awkward fashion through  the UINT sized public interface for read, write, seek, etc.

    It is able to manipulate 4GB+ memory buffer, except that there's no way to hand as much to it. If you start with an empty file and allow it to perform its own allocations, then I bet it'll be able to grow beyond 4GB. But the OP wants to allocate memory on the side, and then have CMemFile write into that memory - that's where you hit the problem.

    Wednesday, July 27, 2016 11:50 PM
  • It is able to manipulate 4GB+ memory buffer, except that there's no way to hand as much to it. If you start with an empty file and allow it to perform its own allocations, then I bet it'll be able to grow beyond 4GB. But the OP wants to allocate memory on the side, and then have CMemFile write into that memory - that's where you hit the problem.

    I'm don't understand the distinction. 

    As I understand the issue the OP wants to use MapViewOfFile on a 4GB+ file and attach the returned pointer to a CMemFile object.  The UINT sized parameter in the Attach function is a problem.  Earlier I suggested that the OP could derive his own class from CMemFile and provide an implementation of Attach that accepted a SIZE_T variable for the buffer size.  It seems to me that is a viable workaround.  Or am I missing something?


    • Edited by RLWA32 Thursday, July 28, 2016 12:19 AM
    Thursday, July 28, 2016 12:19 AM
  • On 7/27/2016 8:19 PM, RLWA32 wrote:

    It is able to manipulate 4GB+ memory buffer, except that there's no way to hand as much to it. If you start with an empty file and allow it to perform its own allocations, then I bet it'll be able to grow beyond 4GB. But the OP wants to allocate memory on the side, and then have CMemFile write into that memory - that's where you hit the problem.

    I'm don't understand the distinction.

    There are two ways to use CMemFile. You can default-construct one and start writing to it; then it allocates its own memory. I don't believe there's a 4GB limit to how much it can allocate in this case.

    Or you can pass a buffer to the constructor or to Attach; then it writes to that buffer, and doesn't attempt to reallocate it. Due to unfortunate choice of function signatures, there's no way to pass a buffer larger than 4GB in this case, even though the class could probably use it just fine.

    As I understand the issue the OP wants to use MapViewOfFile on a 4GB+ file and attach the returned pointer to a CMemFile object.

    Yes, the OP wants to use method #2, the one that has the problem.

    The UINT sized parameter in the Attach function is a problem.  Earlier I suggested that the OP could derive his own class from CMemFile and provide an implementation of Attach that accepted a SIZE_T variable for the buffer size.  It seems to me that is a viable workaround.  Or am I missing something?

    No you are not. It indeed looks like a viable workaround.

    Thursday, July 28, 2016 12:47 AM
  • Yes, this
        void Attach(BYTE* lpBuffer, SIZE_T nBufferSize, SIZE_T nGrowBytes = 0)
        {
          // Code copied from filemem.cpp CMemFile::Attach
          if (lpBuffer == NULL && nBufferSize != 0)
          {
            AfxThrowInvalidArgException();
          }
    
          ASSERT(m_lpBuffer == NULL);
    
          m_nGrowBytes = nGrowBytes;
          m_nPosition = 0;
          m_nBufferSize = nBufferSize;
          m_nFileSize = nGrowBytes == 0 ? nBufferSize : 0;
          m_lpBuffer = lpBuffer;
          m_bAutoDelete = FALSE;
        }

    is a possible workaround - but it would be nice if the parameter is changed to SIZE_T in all functions / ctor in the future.

    Andreas

    If you believe that my suggestion and the code that I provided is the answer to your question I would appreciate it if you would mark my response as the answer instead of marking your own response that contains a copy of my suggested solution.

    Thursday, July 28, 2016 12:49 PM
  • Thank you very much for the help, the code is in production.

    Andreas

    Thursday, July 28, 2016 5:04 PM