none
OpenMP question: How to handle the case more and more threads are created RRS feed

  • Question

  • I've a class which will create a parallel region in the following way. m_LThreadNum is always equal to my core number. The method Run actually runs in a worker thread launched by _begin_thread_ API.

    Network::Run()
    {
    #pragma omp parallel if(bParallel) num_threads(m_lThreadNum)
    #pragma omp for schedule(dynamic)
    for(long lIndex = m_lRDPModuleIndex + 1; lIndex < (long)m_vecModules.size(); lIndex ++)
    {

    m_Error = m_pThisDomain.GetComPtr()->RunModule(lIndex, dStopTime);
    }
    // end of parallel for
    }

    Each time I create this object, and then call its Run method. My core number is 8. Then I noticed each time 8 threads will be created and left there. So, the number of the threads will be more and more. What I'm using is Microsoft VS 2008.

    I don't know how I can control the destroy of the threads in OpenMP. Suppose we don't need to care for this, right?
    Monday, August 30, 2010 10:34 AM

All replies

  • Hi Pengfei,

    Actually, the underlying of the OpenMP is windows thread pool. We can set the thread count by calling num_threads as following:
    http://msdn.microsoft.com/en-us/library/hsa6s00t(VS.80).aspx

    Hence it is unnecessary to care about the thread releasing, the windows thread pool will do that automatically. You could take a look at the document below to get more about OpenMP:
    http://msdn.microsoft.com/en-us/magazine/cc163717.aspx

    Let me know if this helps or not.
    Aland Li

    MSDN Subscriber Support in Forum
    If you have any feedback on our support, please contact msdnmg@microsoft.com


    Please mark the replies as answers if they help and unmark if they don't. This can be beneficial to other community members reading the thread.
    Tuesday, August 31, 2010 1:45 PM
    Moderator
  • Yes, we can set the thread count by calling num_threads. But as you can see from my code snippet, I've used num_threads in the omp directive. And for a parallel region, it is indeed only to create the expected number of threads (in my case, it is 8).

    However, my problem is:

    My Network object internally will launch a new worker thread, and in this worker thread, it will go through the parallel region. So each time I create a Network, then it launch a new worker thread, then it will create a new Omp thread team (ie. 8 threads). And after debugging, I found all of the threads will be left there and not destroyed (at least, I didn't find when they will be destroyed). And my application is a long-live program, in its lifecycle, the sequence of (new Network -> new worker thread -> new Omp thread team) will be repeated many times. So, as you can see, the total number of Omp threads will be very big finally. Actually, this is kind of leak.

    You mentioned, "...Hence it is unnecessary to care about the thread releasing, the windows thread pool will do that automatically". This is also my understanding originally. However, it seems the "automation" didn't work very well.

    Here are two links where this issue is discussed also and actually gave a very clear and detailed explanation for this issue:

    Terminate OpenMP threads explicitly

    http://openmp.org/forum/viewtopic.php?f=3&t=907&start=0&hilit=microsoft

    Intel OpenMP libraries with Visual Studio

    http://openmp.org/forum/viewtopic.php?f=3&t=911&start=0&hilit=microsoft

     

    Actually, I think this might be an issue of OpenMP implementation in Visual Studio. It destroy all the theads only when the process exits. Why not when the master thread exists? And if an additional API is provided to manually destroy a omp thread team (even if this is not demanded by omp specification) will be appreciated.

    Thursday, September 2, 2010 2:30 AM
  • Hi Pengfei,

    Thanks for providing so much information. I think I understand your issue clearly now. Actually, the deedback below talks about the similar issue:
    https://connect.microsoft.com/VisualStudio/feedback/details/340292/openmp-leaks-thread-handels-and-memory-when-used-in-threads?wa=wsignin1.0

    I will continue working on this issue and reply if there are some good news or solutions.

    Regards,
    Aland Li

    MSDN Subscriber Support in Forum
    If you have any feedback on our support, please contact msdnmg@microsoft.com


    Please mark the replies as answers if they help and unmark if they don't. This can be beneficial to other community members reading the thread.
    Thursday, September 2, 2010 12:50 PM
    Moderator
  • Hi, Aland

    Thanks!

    If the issue cannot be resolved in a short time, do you think whether there is some way to workaround this? yesterday, I made an endurance test, and when my thread number is more than 500, my program crashes. This is not accepted for me. I need to figure out a solution as soon as possible.

    Pengfei

    Friday, September 3, 2010 2:37 AM
  • Hi Pengfei,

    After researching on the issue for two days and testing some samples, I reproduced the issue and find some solutions.

    You said: The method Run actually runs in a worker thread launched by _begin_thread_ API.
    So it will create lots of threads and each thread will call the Run method to create more threads. This is the root cause.

    There are two solutions based on my understanding:
    1. Do not call _begin_thread to create threads, use OpenMP directives in all your code snippets. In this case, only the most outer omp parallel block will create multiple threads, the nested omp parallel will only create one thread. You can take a look at the code snippet below:

    	//we use omp directive instead of creating threads.
    	//In this case, only the most outer block will be paralleled.
    	//The nested parallel directives will be executed in one thread.
    	//So the code snippet below will create 1000 threads at most.
    #pragma omp parallel num_threads(1000)
    	{
    #pragma omp for
    		for (int i = 0; i < 50000; ++i) {		
    			Network network(i,1000);
    			//the omp directive in the Run method will only create one thread 
    			network.Run();
    			printf("\n");
    		}
    	}
    

    Since the Run method calling will not create multiple threads in this case, it might not satisfy you. Do not worry, you can follow the second solution.

    2. This solution is different from the first method. It will not case about whether the Run method callings are in different threads, but use a lock to make sure only one Run method calling will be executed anytime. The code snippet shows my idea in detail:

    #include <stdio.h>
    #include <omp.h>
    
    bool bParallel = true;
    
    class Network
    {
    public:
    	Network(int id, long threadNum):
    	 m_iId(id), m_lThreadNum(threadNum) {}
    
    	void Run()
    	{		
    #pragma omp parallel if(bParallel) num_threads(m_lThreadNum)
    		{
    #pragma omp for schedule(dynamic)
    			for(long lIndex = 0; lIndex < 20000; lIndex ++)
    			{
    				//simulate the job processing
    				printf("Network Id:%d, job index: %d\n", m_iId, lIndex);
    			}
    		}
    	}
    private:
    	int m_iId;
    	long m_lThreadNum;
    };
    
    //used to make sure only one Run method is executing anytime.
    omp_lock_t my_lock;
    
    int main()
    {
    	//enable nested parallel to simulate your case.
    	omp_set_nested(1);
    
    	omp_lock_t* pLock = &my_lock;
    	//call this before any Run method is calling
    	omp_init_lock(pLock);
    
    #pragma omp parallel num_threads(1000)
    	{
    		#pragma omp for
    		for (int i = 0; i < 50000; ++i) {
    			omp_set_lock(pLock);			
    			Network network(i,1000);	
    			//Since we lock my_lock in the 
    			network.Run();
    			printf("\n");
    			omp_unset_lock(pLock);
    		}
    	}
    
    	//call this after all the Run methods finish executing
    	omp_destroy_lock(pLock);
    }
    

    Let me know if this helps or not.
    Aland Li

    MSDN Subscriber Support in Forum
    If you have any feedback on our support, please contact msdnmg@microsoft.com

     


    Please mark the replies as answers if they help and unmark if they don't. This can be beneficial to other community members reading the thread.
    Saturday, September 4, 2010 3:02 PM
    Moderator
  • Hi,

    I am writing to check the status of the issue on your side. Could you please let me know if the suggestion works for you? If you have any questions or concerns, please feel free to let me know. I will be more than happy to be of assistance.

     

    Regards,
    Aland Li

    MSDN Subscriber Support in Forum
    If you have any feedback on our support, please contact msdnmg@microsoft.com


    Please mark the replies as answers if they help and unmark if they don't. This can be beneficial to other community members reading the thread.
    Monday, September 6, 2010 3:47 AM
    Moderator
  • Hi, Aland

    Thanks for you great effort. But I think the proposed two solutions are not fit for me.

    Let me explain my application more. My Network includes two threads: 1) main thread - to respond UI; 2) worker thread (created by begin_thread_ex API) to process data. I cannot pre-know how many Networks run. Each time when the user click one UI button, I will create a new Network object and configure it, then it will launch a new thread by begin_thread_ex API and leave the main thread for UI response. In the worker thread, there is actually a endless while loop, and with a fixed time interval, Run method will be called in this while loop. Then, in the Run method, I use OpenMP to process a for loop (actually different dataflow modules in the Network).

    As you can see, the logic is fairly complex, so I cannot easily change the basic code structure.

    Recently, with some help, I've got one workaround for this issue:

    I used omp_set_dynamic(1), and it really made the thread leak much lesser than before.

    Orginally, each time after I create a new Network and Run, there will be 8 threads left (my machine has 8 cores). After using omp_set_dynamic, in many cases, the thread number will kept as 8. That's to say, these 8 threads are reused! However, there is indeed some light-weight leak. That's to say, if I create a new Network and Run it, and repeat to do so several times, the total thread number might be a little more than before (1~2 threads are more).

    I'm very confused by this fact. However, the situation is much better. So, I will fist use this solution.

    However, I hope you can help me on these two questions:

    1. why there are still some light leak? how can we eliminate it?

    2. why omp_set_dynamic(1) can work? I cannot understand this API.

    Thanks a lot!

    Tuesday, September 7, 2010 2:31 AM
  • Hi Pengfei,

    Actually, you can also choose to not modify the code structure. My second solution can also work if the code structure is not adjusted. You could place the code related to lock in the Network class and Run method. The field my_lock can be a static member of the Network class and the method calling of set or reset the lock can be called inside the Run method. All the code except the Network class do not need to be adjusted.

    1. why there are still some light leak? how can we eliminate it?
    Do you mean why the threads are not released in time? If so, the issue might be related to network resources based on my understanding. Anyway, this depends on the code in the Run method. Since I cannot know what the code indeed do from your description, so I cannot show a accurate explanation, but it is mostly related to resouce allocation. However, all such issues can be solved using a lock similar to my second solution.

    2. why omp_set_dynamic(1) can work? I cannot understand this API.
    When we call this method, the number of thread used for executing the code in the parallel region will be automatically adjusted by the runtime. It is hard to say how the underlying implementation works without researching on the source code, but we can make sure it will optimize the code, so the unnecessary threads will not be created. You could take a look at the document below to get more:
    http://msdn.microsoft.com/en-us/library/chy4kf8y(v=VS.80).aspx

    Regards,
    Aland Li

    MSDN Subscriber Support in Forum
    If you have any feedback on our support, please contact msdnmg@microsoft.com


    Please mark the replies as answers if they help and unmark if they don't. This can be beneficial to other community members reading the thread.
    Tuesday, September 7, 2010 6:10 AM
    Moderator