locked
Mutex or Semaphore help wanted RRS feed

  • Question

  • OK, this is the scoop. I have this crazy project that is based on MFCaptureD3D from github. The original project deals with output from a webcam and puts it on a screen. It is a high quality picture. I found a place in device.cpp to intercept the pixel values and my goal is to process these pictures electronically, independently of what MFCaptureD3D is doing. The webcam picture has width 320 pixels and hight - 240 pixels, therefore the total number of pixels is 76,800. The logic of the project demands that those pixels should be divided in small squares each containing about 88 pixels, theremofer the number of such squares whould be 894.

    The problem I have is that taking the pixel values and storing them is a very fast process and my part that should process them is very slow, perhaps a couple of orders of magnitudes slower, q don't really know for sure. In order to try to manage the pixel values coming over at such a speed, I decided to use 3 storage area and the code should fill them sequentually. I hoped that my code will handle such a delay. This is the code to collect the pixel values. pcom is a structure. To fill one array like pcom.S0 for instance it takes less than 7 milliseconds.

                    counter += 1;
                    switch (pcom.rFlag) {
                    case 0:                    
                        pcom.s00[counter] = y0;
                        if (counter == 76799)
                        {                                           
                            pcom.rFlag = 1;
                            counter = -1;
                        }
                        break;
                    case 1:
                        pcom.s01[counter] = y0;
                        if (counter == 76799)
                        {
                            pcom.rFlag = 2;
                            counter = -1;
                            break;
                        }
                        break;
                    case 2:
                        pcom.s02[counter] = y0;
                        if (counter == 76799)
                        {
                            pcom.rFlag = 0;
                            counter = -1;
                            break;
                        }
                        break;
                    }

    Then there is this structure:

    struct OutMemStream2 {	
    	struct A1 {		
    	  struct W1 {
    	    struct C1 {
    	          float x1 = 0.0;   
    		  float y1 = 0.0;  
    		} c[5];                
    	   } w[10];                   
    	} a[90];  
    	int numb_a90 = -1;
    	float invariant = 0;
    }	b[864];  // Expected number of spherical rectangles total 

    The structure is so complicated because the values x1,y1 are in fact functions of a namber of variables, some of which are digital indices. The part of the code that I want to post now is supposed to read raw pixel amplitudes from arrays pcom.S00, pcom.S01, pcom.S02 sequentially, process them and store the results x1,y1 in the structure above.

    for (int ii = 0; ii < pcom.H; ii++)          // pcom.H == 240
        {
            for (int jj = 0; jj < pcom.W; jj++)      // pcom.W == 320
            {
                pixelCounter += 1;
                if (pixelCounter == 0)
                {
                    start = std::chrono::system_clock::now();
                }
                DefineVars(diag, ii, jj, theta, phi);
    
                // determine one of 864 spherical rectangles where the pixel in question belongs to
                rectParalCoo = phi / 0.0872665;  // 0.0872665 == 5 degrees in radians
                colNum = std::floor(rectParalCoo);
                rectMeridCoo = theta / 0.0872665;
                rowNum = std::floor(rectMeridCoo);
                int bNumber = rowNum * 72 + colNum;  // bNumber is the sequential number of the spherical rect.
                b[bNumber].numb_a90 += 1;
    
               std::cout << "   " << bNumber << "   " << b[bNumber].numb_a90 << "       " << pcom.rFlag << endl;
    
                // just determined theta & phi angles for individual pixels
                // find Spherical Harmonic for the pixels and to pull pixel's amplitude
    
                switch (pcom.rFlag){
                case 0:
                    value = pcom.s01[pixelCounter]; <== VERY FAST
                    break;
                case 1:
                    value = pcom.s02[pixelCounter]; 
                    break;
                case 2:
                    value = pcom.s00[pixelCounter]; 
                    break;
                }  
                llCou = -1;
                for (int ll = pcom.llMin; ll < pcom.llMax; ll++)       // 10 values of ll
                {
                    llCou += 1;
                    mmCou = -1;  // VERY SLOW PART
                    for (int mm = pcom.mmMin; mm <= pcom.mmMax; mm++)  // 5 values of mm
                    {
                        mmCou += 1;
                        res = (double)LegendrePolynomials::normSelector(cos(theta), ll, mm);
                        res *= pcom.factorArr[(int)diag];  // peripheral attenuation/modulation
                        res *= (float)value;  //  int value is the amplitude of signal at a pixel
                        // selection of the pixel input point by point                                          
                        BytesWritten += sizeof(float);  
                        b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].x1 = res * cos(phi * (float)mm);
                        b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].y1 = res * sin(phi * (float)mm);
                //        pfRArray [pixelCounter] = res * cos(phi * (float)mm); // real part
                //        pfIArray [pixelCounter] = res * sin(phi * (float)mm); // imaginary part                     
                    }
                }            
            }
        }

    It is a simple piece of code, however it does not work because of the difference in speed. I run into exceptions and the pixels end up all being messed up. I know where the solution lies. I need to use either a mutex or a semaphore, but I don't know how to attach them to my code. I would appreciate if somebody will offer a helping hand.

    Thank you - MyCatAlex

    Wednesday, June 10, 2020 6:04 PM

Answers

  • Well, the need for a synchronization object stems from using more than one thread (or process) to access a shared resource.  Is that what your code is doing?

    The difference between a mutex and a sempahore are in the explanations provided at Synchronization Objects.  A less expensive alternative to a mutex (which is a kernel object) that provides synchronization between the threads of a single process are  Critical Section Objects

    • Marked as answer by MyCatAlex Thursday, June 11, 2020 8:22 PM
    Wednesday, June 10, 2020 9:12 PM
  •  First thing I want to do is to place the structure OutMemStream2 to an SSD disk. I've tried some ideas but nothing works. The project does not recognize it.

    If you are talking about using a memory-mapped file for data to be stored in an array of OutMemStream2 structures then the principle is the same as using an array of floats.

    The maximum file size should be 864 * sizeof(OMemStream2);

    The mapped view would be -

    OMemStream2 *poms2 = (OMemStream2*) MapViewOfFile(..);

    An individual OMemStream2 structure in the array would be referenced using an array index from 0 to 863 -

    poms2[array index]


    // b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].x1 = res * cos(phi * (float)mm); // b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].y1 = res * sin(phi * (float)mm); pfRArray [pixelCounter] = res * cos(phi * (float)mm); // real part

    pfIArray [pixelCounter] = res * sin(phi * (float)mm); // imaginary part

    The method that you've given to me works and writes all pixel values in place. The code that I commented out runs into an exception very soon, it does not work, but I prefer to write in a structure, not an a simplified array.

    Well, just saying that the commented code "runs into an exception" is not informative.

    Are you sure that the values that you are using to index the various arrays are valid?  I suggest you consider including some assertions in the code to validate the data being used.

    If in fact you do need to make access to a shared resource thread-safe then I doubt that chrono sleep is a reliable solution.  There is no guarantee that one thread will complete its use of the resource while other threads that share it are sleeping.

    Thank you. I do have my doubts too. Although the project appears to be working now, still there is a huge discrepancy in execution time. The execution time of the block that I have posted and which is supposed to read all the pixel values stored (76,800) is 111.0 seconds, totally unacceptable. The execution time of the previous block that writes them to an array is under 7 milliseconds.

    Since you don't show exactly how you are timing code execution I can't comment on your observations.  Regardless, 111 seconds sounds odd.

    ----------------------------------------------------

    And all of the above is without regard to multi-threading considerations and protecting access to shared data, none of which can be discerned from what you have shown so far.




    • Edited by RLWA32 Saturday, June 13, 2020 4:53 PM typos
    • Marked as answer by MyCatAlex Saturday, June 13, 2020 5:12 PM
    Saturday, June 13, 2020 4:39 PM
  • By using

    #include "Blocks864.cpp"

    the compiler is inserting the declaration/definition of the array directly into the compilation.

    Get rid of the Common.h header inclusion of

    struct OutMemStream2 {}b;

    • Marked as answer by MyCatAlex Sunday, June 14, 2020 12:40 PM
    Sunday, June 14, 2020 12:10 PM
  • I can't tell from what was posted exactly the file placement of the #include "Blocks864.cpp" statement in relation to other code but the declaration of the OutMemStream2 structure had not been made when the compiler encountered the indicated statements.

    I suggest you change your approach.

    Include the declaration of your OutMemStream2 structure in a header file.

    For example, in your Common.h header -

    #pragma once
    
    struct OutMemStream2 {
    	struct A1 {
    		struct W1 {
    			struct C1 {
    				float x1 = 0.0;
    				float y1 = 0.0;
    			} c[5];
    		} w[10];
    	} a[90];
    	int numb_a90 = -1;
    	float invariant = 0;
    };

    Then you can use #include "Common.h" in your code to make the OutMemStream2 structure type available.

    Remove the #include "Blocks864.cpp" statement.

    If you want to work with a statically allocated array of OutMemStream2 structs then you can define a global variable for the array.  Use a global variable because an array of 864 OutMemStream2 structures is too large for the default stack size.

    For example, outside of any function definition -

    OutMemStream2 b[864];

    It also appears that you have included your own declaration of the _beginthreadex function in your own source.  You should be using the standard headers for this purpose.

    I have not looked at the file mapping code.


    • Edited by RLWA32 Sunday, June 14, 2020 2:55 PM
    • Marked as answer by MyCatAlex Sunday, June 14, 2020 3:00 PM
    Sunday, June 14, 2020 2:54 PM
  • I cannot understand what it is you are doing or what code you are actually working with.

    The descriptions provided have become convoluted and I am now totally confused.

    • Marked as answer by MyCatAlex Sunday, June 14, 2020 4:24 PM
    Sunday, June 14, 2020 3:39 PM
  • What can I do to make it clear to you?

    I do not have a system with a webcam so I cannot execute the code.

    Seeing snippets of code that are without context doesn't help me understand what is happening.  Maybe I'm just slow.

    If you upload your project to OneDrive (or equivalent) then I could download it and open the project on my own system.  Seeing how all the pieces fit together would help me to understand.

    • Marked as answer by MyCatAlex Sunday, June 14, 2020 7:48 PM
    Sunday, June 14, 2020 5:52 PM
  • I meant the entire solution/project with all files so that I can build it on my own system.
    • Marked as answer by MyCatAlex Sunday, June 14, 2020 10:14 PM
    Sunday, June 14, 2020 9:54 PM

All replies

  • Well, the need for a synchronization object stems from using more than one thread (or process) to access a shared resource.  Is that what your code is doing?

    The difference between a mutex and a sempahore are in the explanations provided at Synchronization Objects.  A less expensive alternative to a mutex (which is a kernel object) that provides synchronization between the threads of a single process are  Critical Section Objects

    • Marked as answer by MyCatAlex Thursday, June 11, 2020 8:22 PM
    Wednesday, June 10, 2020 9:12 PM
  • Well, the need for a synchronization object stems from using more than one thread (or process) to access a shared resource.  Is that what your code is doing?

    The difference between a mutex and a sempahore are in the explanations provided at Synchronization Objects.  A less expensive alternative to a mutex (which is a kernel object) that provides synchronization between the threads of a single process are  Critical Section Objects

    Thank you. It is a valid suggestion, however, I managed to do what I wanted with chrono sleep function. It works well, but still there are problems. Had this idea come to my mind a day earlier, I would not have posted the OP. Now, my step is to move that telescopic structure to an external SSD disk and start setting up multiple threads for parallel processing.

    Yes, I believe more than one thread is accessing a shared resource in my project.

    Thank you, MyCatAlex


    • Edited by MyCatAlex Thursday, June 11, 2020 8:30 PM
    Thursday, June 11, 2020 8:29 PM
  • If in fact you do need to make access to a shared resource thread-safe then I doubt that chrono sleep is a reliable solution.  There is no guarantee that one thread will complete its use of the resource while other threads that share it are sleeping.
    Thursday, June 11, 2020 9:36 PM
  • If in fact you do need to make access to a shared resource thread-safe then I doubt that chrono sleep is a reliable solution.  There is no guarantee that one thread will complete its use of the resource while other threads that share it are sleeping.

    Thank you. I do have my doubts too. Although the project appears to be working now, still there is a huge discrepancy in execution time. The execution time of the block that I have posted and which is supposed to read all the pixel values stored (76,800) is 111.0 seconds, totally unacceptable. The execution time of the previous block that writes them to an array is under 7 milliseconds. First thing I want to do is to place the structure OutMemStream2 to an SSD disk. I've tried some ideas but nothing works. The project does not recognize it. This is how the code looks now:

    // b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].x1 = res * cos(phi * (float)mm); // b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].y1 = res * sin(phi * (float)mm); pfRArray [pixelCounter] = res * cos(phi * (float)mm); // real part

    pfIArray [pixelCounter] = res * sin(phi * (float)mm); // imaginary part

    The method that you've given to me works and writes all pixel values in place. The code that I commented out runs into an exception very soon, it does not work, but I prefer to write in a structure, not an a simplified array.

    If I am able to place the structure in an SSD and force it working (accepting individual values), I will see how it improved the execution time and if not, then I will have to go to Azure and do multithreading approach.

    Could you help me to place the structure in SSD in such a way that the program will accept it as a header file, for example?

    You probably see in the code, that each pixel is associated with 50 values of special functions. Thus the total values of meaningful variables for this project to handle is 76800 x 50 = 3,840,000. It is much easier to store them all in a structure than in individual one-dim arrays.

    Thank you, - MyCatAlex



    • Edited by MyCatAlex Friday, June 12, 2020 6:53 PM
    Friday, June 12, 2020 2:56 PM
  •  First thing I want to do is to place the structure OutMemStream2 to an SSD disk. I've tried some ideas but nothing works. The project does not recognize it.

    If you are talking about using a memory-mapped file for data to be stored in an array of OutMemStream2 structures then the principle is the same as using an array of floats.

    The maximum file size should be 864 * sizeof(OMemStream2);

    The mapped view would be -

    OMemStream2 *poms2 = (OMemStream2*) MapViewOfFile(..);

    An individual OMemStream2 structure in the array would be referenced using an array index from 0 to 863 -

    poms2[array index]


    // b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].x1 = res * cos(phi * (float)mm); // b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].y1 = res * sin(phi * (float)mm); pfRArray [pixelCounter] = res * cos(phi * (float)mm); // real part

    pfIArray [pixelCounter] = res * sin(phi * (float)mm); // imaginary part

    The method that you've given to me works and writes all pixel values in place. The code that I commented out runs into an exception very soon, it does not work, but I prefer to write in a structure, not an a simplified array.

    Well, just saying that the commented code "runs into an exception" is not informative.

    Are you sure that the values that you are using to index the various arrays are valid?  I suggest you consider including some assertions in the code to validate the data being used.

    If in fact you do need to make access to a shared resource thread-safe then I doubt that chrono sleep is a reliable solution.  There is no guarantee that one thread will complete its use of the resource while other threads that share it are sleeping.

    Thank you. I do have my doubts too. Although the project appears to be working now, still there is a huge discrepancy in execution time. The execution time of the block that I have posted and which is supposed to read all the pixel values stored (76,800) is 111.0 seconds, totally unacceptable. The execution time of the previous block that writes them to an array is under 7 milliseconds.

    Since you don't show exactly how you are timing code execution I can't comment on your observations.  Regardless, 111 seconds sounds odd.

    ----------------------------------------------------

    And all of the above is without regard to multi-threading considerations and protecting access to shared data, none of which can be discerned from what you have shown so far.




    • Edited by RLWA32 Saturday, June 13, 2020 4:53 PM typos
    • Marked as answer by MyCatAlex Saturday, June 13, 2020 5:12 PM
    Saturday, June 13, 2020 4:39 PM
  • If you are talking about using a memory-mapped file for data to be stored in an array of OutMemStream2 structures then the principle is the same as using an array of floats.

    Yes, I want to use MemoryMappedFiles. I will try to develop what you just showed, will see how it works. If it does not I will try to show you more code. The trouble with showing more code is that there is a lot of it, some parts are temporary or irrelevant, but I will try my best. Some snipping is needed.

    Thank you, - MyCatAlex


    Saturday, June 13, 2020 5:18 PM
  • I read your post again and realized the you were interested in how I measured time. This is simple.

    std::chrono::time_point<std::chrono::system_clock> start, endB;
    ....................................
    if (pixelCounter == 0)
                {
                    start = std::chrono::system_clock::now();
                }
    ..........................................
        endB = std::chrono::system_clock::now();
        std::chrono::duration<double> elapsed_seconds = endB - start;
        std::cout << "time count " << elapsed_seconds.count() << endl;

    The first statement is somewhere at the beginning of a routine that reads pixel values and multiplies them by special functions, etc. The last block is after that routine finished execution.

    Thank you, - MyCatAlex

    Saturday, June 13, 2020 5:27 PM
  • OK, this is what I did:

    I had structure OutMemStream2 stored in a header file Commons.h. I commented it out, instead I left in place just a definition:

    struct OutMemStream2 {}b;

    I also created a file "Blocks864.cpp" and included it in the project as a source file. It has pragma once and the structure, nothing else.

    #pragma once
    struct OutMemStream2 {	
    	struct A1 {		
    		struct W1 {
    			struct C1 {
    				float x1 = 0.0;    // pixel amplitude multiplied by Re of function
    				float y1 = 0.0;    // pixel amplitude multiplied by Im of function
    			} c[5];             // all mm value's (5) for a given ll
    		} w[10];                // 10 different ll indices
    	} a[90];  // 90 substructures corresponding expected number of pixels per spherical rectangle
    	int numb_a90 = -1;
    	float invariant = 0;
    }	b[864];  // Expected number of spherical rectangles total

    I also included this file Blocks864.cpp among the #include files in device.cpp which is the major my source file now since I added so much other source code to it, some of that doing nothing since plans keep changing. This #include looks like this:

    #include "Blocks864.cpp"

    A compilation attempt gave 12 errors all more or lest similar. These pieces of code where they happened:

    1316  if (b[bNumber].numb_a90 < 90)
    1317      b[bNumber].numb_a90 += 1;
    
    Then:
    
    1348  b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].x1 = res * cos(phi * (float)mm);
    1349 b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].y1 = res * sin(phi * (float)mm);

    These are the errors:

    Severity	Code	Description	Project	Path	File	Line	Suppression State
    Error	C2676	binary '[': 'OutMemStream2' does not define this operator or a conversion to a type acceptable to the predefined operator	MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D\device.cpp	1316	
    
    

    The remaining 11 errors are related to other square brackets. There are few of them. Something is probably wrong with the way I defined the structure.

    Thank you, - MyCatAlex


    • Edited by MyCatAlex Saturday, June 13, 2020 9:53 PM
    Saturday, June 13, 2020 7:16 PM
  • By using

    #include "Blocks864.cpp"

    the compiler is inserting the declaration/definition of the array directly into the compilation.

    Get rid of the Common.h header inclusion of

    struct OutMemStream2 {}b;

    • Marked as answer by MyCatAlex Sunday, June 14, 2020 12:40 PM
    Sunday, June 14, 2020 12:10 PM
  • By using

    #include "Blocks864.cpp"

    the compiler is inserting the declaration/definition of the array directly into the compilation.

    Get rid of the Common.h header inclusion of

    struct OutMemStream2 {}b;

    Thank you. 14 errors on compilation. They are mostly in this block. Just a comment. I have 3 or even 4 blocks of separate Memory-Map-File definition, two of them are from what I tried to do in RAM, one (probably still there) is your demo and the last one is the definition placed in SSD. I had to invent new memorable abbreviations. Here is the code:

    1242  dwMaxSize = 864 * sizeof(OutMemStream2);
        HANDLE hR_File = CreateFile(_T("Blocks864.cpp"), GENERIC_READ | GENERIC_WRITE, 0, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, NULL);
        if (hR_File == INVALID_HANDLE_VALUE)
        {
            _tprintf_s(_T("CreateFile hR_File (real part) failed with error %d\n"), GetLastError());
            return;
        }
        HANDLE hR_Map = CreateFileMapping(hR_File, NULL, PAGE_READWRITE, 0, dwMaxSize, NULL);
    1250    OutMemStream2* poms2 = (OutMemStream2*)MapViewOfFile(hR_Map, FILE_MAP_ALL_ACCESS, 0, 0, 0);
        _tprintf_s(_T("Maximum mapped memory is %u bytes, file size on disk is now %u bytes\n"),
            dwMaxSize, GetFileSize(hR_File, NULL));
        if (!hR_Map)
        {
            _tprintf_s(_T("CreateFileMapping failed with error %d\n"), GetLastError());
            return;
        }
        MEMORY_BASIC_INFORMATION mbi_Q{};
        if (VirtualQuery(pfRArray, &mbi_Q, sizeof mbi_R))
        {
            _tprintf_s(_T("Virtual memory for mapped view --\n\tBase address - 0x%p\n\tEnd address - 0x%p\n\tRegion size - %Id bytes\n"),
                mbi_Q.BaseAddress,
                (LPVOID)((DWORD_PTR)mbi_Q.BaseAddress + mbi_Q.RegionSize - 1),
                mbi_Q.RegionSize);
        }
    
        // Microsoft recommends protecting reading/writing from memory mapped files
        // with structured exception handling
        // See https://docs.microsoft.com/en-us/windows/win32/memory/reading-and-writing-from-a-file-view
    
        _ACRTIMP uintptr_t __cdecl _beginthread(
            _In_     _beginthread_proc_type _StartAddress,
            _In_     unsigned               _StackSize,
            _In_opt_ void* _ArgList
        );


     Errors 


    Severity	Code	Description	Project	Path	File	Line	Suppression State
    Error	C2065	'OutMemStream2': undeclared identifier	MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D\device.cpp	1250	
    
    Error	C2065	'poms2': undeclared identifier	MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D\device.cpp	1250	
    
    Error	C2059	syntax error: ')'	MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D\device.cpp	1250	
    
    Error	C2065	'b': undeclared identifier	MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D	C:\CPlusPlus_Projects\MFCaptureD3D\MFCaptureD3D\device.cpp	1316	
    
    

    The rest of them are repetitious. Also while I typed this the number of errors "magically" increased to 21.

    Thank you, - MyCatAlex


    • Edited by MyCatAlex Sunday, June 14, 2020 12:57 PM
    Sunday, June 14, 2020 12:56 PM
  • I can't tell from what was posted exactly the file placement of the #include "Blocks864.cpp" statement in relation to other code but the declaration of the OutMemStream2 structure had not been made when the compiler encountered the indicated statements.

    I suggest you change your approach.

    Include the declaration of your OutMemStream2 structure in a header file.

    For example, in your Common.h header -

    #pragma once
    
    struct OutMemStream2 {
    	struct A1 {
    		struct W1 {
    			struct C1 {
    				float x1 = 0.0;
    				float y1 = 0.0;
    			} c[5];
    		} w[10];
    	} a[90];
    	int numb_a90 = -1;
    	float invariant = 0;
    };

    Then you can use #include "Common.h" in your code to make the OutMemStream2 structure type available.

    Remove the #include "Blocks864.cpp" statement.

    If you want to work with a statically allocated array of OutMemStream2 structs then you can define a global variable for the array.  Use a global variable because an array of 864 OutMemStream2 structures is too large for the default stack size.

    For example, outside of any function definition -

    OutMemStream2 b[864];

    It also appears that you have included your own declaration of the _beginthreadex function in your own source.  You should be using the standard headers for this purpose.

    I have not looked at the file mapping code.


    • Edited by RLWA32 Sunday, June 14, 2020 2:55 PM
    • Marked as answer by MyCatAlex Sunday, June 14, 2020 3:00 PM
    Sunday, June 14, 2020 2:54 PM
  • Thank you RLWA32. I just made a simple change. I uncommented the structure //struct OutMemStream2 {}b; and commented out the structure definition that I just posted, both in Commons.h. To my surprise the code compiled! However  after it began running it generated an exception (access violation) in this statement

     llCou = -1;
                for (int ll = pcom.llMin; ll < pcom.llMax; ll++)       // 10 values of ll
                {
                    llCou += 1;
                    mmCou = -1;
                    for (int mm = pcom.mmMin; mm <= pcom.mmMax; mm++)  // 5 values of mm
                    {
                        mmCou += 1;
                        res = (double)LegendrePolynomials::normSelector(cos(theta), ll, mm);
                        res *= pcom.factorArr[(int)diag];  // peripheral attenuation/modulation
                        res *= (float)value;  //  int value is the amplitude of signal at a pixel
                        BytesWritten += sizeof(float);  
     //   std::cout << " bNumber " << bNumber << "  numb_a90  " << b[bNumber].numb_a90 << endl;
                        b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].x1 = res * cos(phi * (float)mm);
                        b[bNumber].a[b[bNumber].numb_a90].w[llCou].c[mmCou].y1 = res * sin(phi * (float)mm); <== Exception
    
                   //     pfRArray [pixelCounter] = res * cos(phi * (float)mm); // real part
                   //     pfIArray [pixelCounter] = res * sin(phi * (float)mm); // imaginary part                     
                    }
                }            

    When checked for values after the exception is thrown, it becomes clear that this value

    b[bNumber].numb_a90

    is absurdly large, but it appeared only after the previous statement was executed! This value has limits between 0 and 89. I have a std::out statement previously which is commented out and I used it many times trying to find out where that large value comes from - no success.

    That statement to start a thread I copied from your demo. I think it was a part of it. Every time I wrote code for Memory-Map-File, I included it.

    Just to reiterate, I included this code:

    #pragma once
    struct OutMemStream2 {	
    	struct A1 {		
    		struct W1 {
    			struct C1 {
    				float x1 = 0.0;    // pixel amplitude multiplied by Re of function
    				float y1 = 0.0;    // pixel amplitude multiplied by Im of function
    			} c[5];             // all mm value's (5) for a given ll
    		} w[10];                // 10 different ll indices
    	} a[90];  // 90 substructures corresponding expected number of pixels per spherical rectangle
    	int numb_a90 = -1;
    	float invariant = 0;
    }	b[864];  // Expected number of spherical rectangles total

    in Blocks864.cpp file which I created manually in an SSD 1 TB file. If necessary, I increase the SSD memory, quadruple it, but I think it sort of worked before.

    If you want to work with a statically allocated array of OutMemStream2 - it was my intent from the beginning to work with a statically allocated array... 

    Thank you. I appreciate your help. I have to read your previous post again to find out what that meant.

    - MyCatAlex



    • Edited by MyCatAlex Sunday, June 14, 2020 3:37 PM
    Sunday, June 14, 2020 3:32 PM
  • I cannot understand what it is you are doing or what code you are actually working with.

    The descriptions provided have become convoluted and I am now totally confused.

    • Marked as answer by MyCatAlex Sunday, June 14, 2020 4:24 PM
    Sunday, June 14, 2020 3:39 PM
  • I just removed the #include "Blocks864.cpp" statement. Nothing has changed. Compilation + exception almost immediately.
    Sunday, June 14, 2020 3:44 PM
  • I want to give you a work diagram that seems logical to me. The subroutine first determines the spherical coordinates of each pixel by going through them one by one. It is doable and as a result I get two angles: theta and phi - elevation and azimuth. While going trough the sequence of pixels the subroutine also determines one of 864 squares where this particular pixel falls into. So far the routine has dealt with pixel coordinates only. Then it determines the multiplication coefficients for each pixel, they are coordinate dependent too, and the next step is to multiply the amplitudes of signal (brightness) of each pixel by coordinate dependent function values. You can see that some of the values here might be stored "forever" in separate arrays. That will be the next step.

    This is a significant part of it but not everything.

    Thank you, - MyCatAlex

    Sunday, June 14, 2020 4:01 PM
  • I cannot understand what it is you are doing or what code you are actually working with.

    The descriptions provided have become convoluted and I am now totally confused.

    What can I do to make it clear to you?

    - MyCatAlex

    Sunday, June 14, 2020 4:25 PM
  • What can I do to make it clear to you?

    I do not have a system with a webcam so I cannot execute the code.

    Seeing snippets of code that are without context doesn't help me understand what is happening.  Maybe I'm just slow.

    If you upload your project to OneDrive (or equivalent) then I could download it and open the project on my own system.  Seeing how all the pieces fit together would help me to understand.

    • Marked as answer by MyCatAlex Sunday, June 14, 2020 7:48 PM
    Sunday, June 14, 2020 5:52 PM
  • What can I do to make it clear to you?

    I do not have a system with a webcam so I cannot execute the code.

    Seeing snippets of code that are without context doesn't help me understand what is happening.  Maybe I'm just slow.

    If you upload your project to OneDrive (or equivalent) then I could download it and open the project on my own system.  Seeing how all the pieces fit together would help me to understand.

    I'l do it, give me a few hours, the latest will be tomorrow morning. Do you want MSCaptureD3D part also?

    Thank you, - MyCatAlex

    Sunday, June 14, 2020 7:51 PM
  • I meant the entire solution/project with all files so that I can build it on my own system.
    • Marked as answer by MyCatAlex Sunday, June 14, 2020 10:14 PM
    Sunday, June 14, 2020 9:54 PM
  • I meant the entire solution/project with all files so that I can build it on my own system.

    I understand. Tomorrow morning. BTW on VS2017 it will give you 520 or so compile errors. I have no idea how you will handle it. One more thing. The project appeared to be working, sending values for storage in SSD, but it used only a small portion of the struct OutMemStream2, how do I understood it? Because I used a different storage method. Well, I will have to tell about it later. There is code in the project that is not working now. You will see.

    - MyCatAlex


    • Edited by MyCatAlex Monday, June 15, 2020 12:38 PM
    Sunday, June 14, 2020 10:19 PM
  • I meant the entire solution/project with all files so that I can build it on my own system.

    I understand. Tomorrow morning. BTW on VS2017 it will give you 520 or so compile errors. I have no idea how you will handle it. One more thing. The project appeared to be working, sending values for storage in SSD, but it used only a small portion of the struct OutMemStream2, how do I understood it? Because I used a different storage method. Well, I will have to tell about it later. There is code in the project that is not working now. You will see.

    - MyCatAlex


    OK, unless I screwed it up I just uploaded a folder to my onedrive. Since I run Win10, it has already been built in my OS, that's good. I tghink I uploaded the entire folder, but I am not sure it really worked.

    I would like you to acknowledge with a brief post that you are monitoring the thread and after that I will post the link to the onedrive code. After that please acknowledge that you got it and I will remove the link.

    I hope I did not screw anything up which might be expected from a person who had never done it before. I did everything by googling.

    OK, now I cannot sign in. OneDrive says I don't have the access to this service. J was just able to sign in 15 min ago. What a system! Nothing but .....

    What is the alternative to MS OneDrive? This one locked me out and I don't know why. I need to work on it.

    To be locked out of OneDrive is a known problem. I have to resolve it somehow. Somebody is coming up to help me.

    Thanks, - MyCatAlex












    • Edited by MyCatAlex Monday, June 15, 2020 4:33 PM
    Monday, June 15, 2020 2:04 PM
  • I'll post back to this thread tomorrow morning to let you know when I'm active on the forums.

    Tuesday, June 16, 2020 12:22 AM
  • I'll let you know when I have downloaded the project from your onedrive links.
    Tuesday, June 16, 2020 1:04 PM