SetThreadGroupAffinity returning invalid parameter the 2nd time it's used in the code

Question

Hi all,

I'm trying to write some code for handling affinity that use the new functions supporting processor groups. I've got a system running x64 Win 2k8 R2 DataCenter with 48 logical processors. I ran bcdedit.exe /set groupsize 32 to split the 48 procs into 2 groups.

My software is starting multiple threads and setting each thread to a different logical processor.

The first thread that I execute SetThreadGroupAffinity for is successful. On every thread after the first, SetThreadGroupAffinity returns Invalid Parameter.

I simplified the code to do the following (somewhat pseudo):

///////////////
/// Code Example
///////////////

GROUP_AFFINITY previousAffinity;
GROUP_AFFINITY GroupAffinity;
GroupAffinity.Group = 0;
GroupAffinity.Mask = 1;

hthread  = (HANDLE) _beginthreadex(NULL, 0, &cache, (void*) &arglist, CREATE_SUSPENDED, &threadid);
hthread2  = (HANDLE) _beginthreadex(NULL, 0, &cache, (void*) &arglist, CREATE_SUSPENDED, &threadid);

if (SetThreadGroupAffinity(hThread, &GroupAffinity, &previousAffinity) != 0)
	{
            ResumeThread(hthread);
         }
        //else I print GetLastError
if (SetThreadGroupAffinity(hThread2, &GroupAffinity, &previousAffinity) != 0)
	{
            ResumeThread(hthread2); //edit fixed typo
         }
         //else I print GetLastError

Regardless of what order I start the threads or set the affinity of the 2 threads, the first SetThreadGroupAffinity call works, and the 2nd call returns invalid parameter.

I opened the exe from within WinDBG to step through the code and found that NtSetInformationThread is the one returning bad parameter. I compared the passing code flow to the failing code flow and found that it was identical up until this function, so I don't know why it would return bad parameter. Here's the last couple instructions for the good flow and bad flow:

////////////////////

/// Good Flow

///////////////////

0:000> t

ntdll!NtSetInformationThread+0x8:

00000000`7785ff88 0f05            syscall

0:000> r

rax=000000000000000a rbx=000000000012fdb8 rcx=0000000000000038

rdx=000000000000001e rsi=000000000012fe50 rdi=0000000000000038

rip=000000007785ff88 rsp=000000000012fd08 rbp=0000000000000000

 r8=000000000012fe50  r9=0000000000000010 r10=0000000000000038

r11=0000000000000246 r12=0000000000000000 r13=0000000000000000

r14=0000000000000000 r15=0000000000000000

iopl=0         nv up ei pl zr na po nc

cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

ntdll!NtSetInformationThread+0x8:

00000000`7785ff88 0f05            syscall

 

0:000> t

ntdll!NtSetInformationThread+0xa:

00000000`7785ff8a c3              ret

0:000> r

rax=0000000000000000 rbx=000000000012fdb8 rcx=000000007785ff8a

rdx=0000000000000000 rsi=000000000012fe50 rdi=0000000000000038

rip=000000007785ff8a rsp=000000000012fd08 rbp=0000000000000000

 r8=000000000012fd08  r9=0000000000000000 r10=0000000000000000

r11=0000000000000346 r12=0000000000000000 r13=0000000000000000

r14=0000000000000000 r15=0000000000000000

iopl=0         nv up ei pl zr na po nc

cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

ntdll!NtSetInformationThread+0xa:

00000000`7785ff8a c3              ret

////////////////

// Bad Flow

/////////////

0:000> t

ntdll!NtSetInformationThread+0x8:

00000000`7785ff88 0f05            syscall

0:000> r

rax=000000000000000a rbx=000000000012fdb8 rcx=0000000000000040

rdx=000000000000001e rsi=000000000012fe50 rdi=0000000000000040

rip=000000007785ff88 rsp=000000000012fd08 rbp=0000000000000000

 r8=000000000012fe50  r9=0000000000000010 r10=0000000000000040

r11=0000000000000246 r12=0000000000000000 r13=0000000000000000

r14=0000000000000000 r15=0000000000000000

iopl=0         nv up ei pl zr na po nc

cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

ntdll!NtSetInformationThread+0x8:

00000000`7785ff88 0f05            syscall

 

0:000> t

ntdll!NtSetInformationThread+0xa:

00000000`7785ff8a c3              ret

0:000> r

rax=00000000c000000d rbx=000000000012fdb8 rcx=000000007785ff8a

rdx=0000000000000000 rsi=000000000012fe50 rdi=0000000000000040

rip=000000007785ff8a rsp=000000000012fd08 rbp=0000000000000000

 r8=000000000012fd08  r9=0000000000000000 r10=0000000000000000

r11=0000000000000346 r12=0000000000000000 r13=0000000000000000

r14=0000000000000000 r15=0000000000000000

iopl=0         nv up ei pl zr na po nc

cs=0033  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000246

ntdll!NtSetInformationThread+0xa:

00000000`7785ff8a c3              ret

 

The only difference I see is the thread handle parameter, yet in the failing case it puts 0xc000000d in eax.

This code works perfectly fine if I use SetThreadAffinityMask instead, but of course doesn't allow me to schedule threads on other groups.

Let me know if you need more information

 

Answer 1

Hi, I don't believe your issue here is related to the Concurrency Runtime.  You'll likely get a better answer if you post in the Windows SDK forum (http://social.msdn.microsoft.com/Forums/en-US/windowssdk/threads)

However, I did take a quick look, and I wasn't able to reproduce your issue.  I only have a 4-core box here, but I did a "bcdedit.exe /set groupsize 2" to also get 2 groups.  I basically copied your code and both SetThreadGroupAffinity calls succeded for me.  (I did notice one small typo in your above code that I needed to correct -- your SetThreadGroupAffinity calls have an arguments "hThread" and "hThread2" while you created threads "hthread" and "hthread2".  I guess that could conceivably cause your error, but I'll assume that's just a typo in your forum post?)

I posted the code I ran below, you can try and see if it works on your machine.  Otherwise, you might want to try the WinSDK forum.

#include <windows.h>
#include <process.h>
#include <stdio.h>

unsigned __stdcall cache( void* pArguments )
{
    printf( "In thread...\n" );

    _endthreadex( 0 );
    return 0;
} 

int main()
{
    HANDLE hthread, hthread2;
    unsigned threadid;

    GROUP_AFFINITY previousAffinity;
    GROUP_AFFINITY GroupAffinity;
    GroupAffinity.Group = 0;
    GroupAffinity.Mask = 1;

    hthread  = (HANDLE) _beginthreadex(NULL, 0, &cache, NULL, CREATE_SUSPENDED, &threadid);
    hthread2  = (HANDLE) _beginthreadex(NULL, 0, &cache, NULL, CREATE_SUSPENDED, &threadid);

    if (SetThreadGroupAffinity(hthread, &GroupAffinity, &previousAffinity) != 0)
    {
                ResumeThread(hthread);
    }
    else
    {
        printf("error 1\n");
    }

    if (SetThreadGroupAffinity(hthread2, &GroupAffinity, &previousAffinity) != 0)
    {
                ResumeThread(hthread2); //edit fixed typo
    }
    else
    {
        printf("error 2\n");
    }

    WaitForSingleObject( hthread, INFINITE );
    WaitForSingleObject( hthread2, INFINITE );
}

Answer 2

I was originally trying to update an app written in C to use some C++ affinity functions. I put that on hold and started with a new solution written in C++ and was able to get this working. I tried using the same working functions with the C code and still hit the stated problem. When I have time to get back to it, I'll post some specific code that fails. Thanks!

Answer 3

I've had time to get back to this problem again.

It has to do with optimization. The code below fails. If I uncomment the print statement, it works fine. It also works fine without the print statement if I disable optimization.

int _setAffinity(HANDLE hThread, GROUP_AFFINITY GroupAffinity)
{
	GROUP_AFFINITY newGroupAffinity;

	newGroupAffinity.Group = GroupAffinity.Group;
	newGroupAffinity.Mask = GroupAffinity.Mask;
	
	if (SetThreadGroupAffinity(hThread, &newGroupAffinity, NULL) == 0)
	{
		//printf("\r\ndebug1:: hThread: %X, Group: %X, Mask: %X\r\n",hThread, newGroupAffinity.Group, newGroupAffinity.Mask);
		return -1;
	}
	else
	{
		return _getThreadNumber(GroupAffinity);
	}
}

Answer 4

Oops sorry, I clicked the wrong button. What I meant was the sample program given by Mike Chu on Tuesday, January 05, 2010 1:54 AM does not work me. I have a 4 core - 8 logical processor system. I did the bcedit just as Mike did and then ran his program. Both the SetThreadGroupAffinity() calls failed with "Invalid Parameter" - 87 error as returned by GetLastError(). I tried commenting off the printf that did not help. What is invalid of the parameter I dont understand. Can anyone help pelase? 

thanks

Answer 5

I've had time to get back to this problem again.

It has to do with optimization. The code below fails. If I uncomment the print statement, it works fine. It also works fine without the print statement if I disable optimization.

int _setAffinity(HANDLE hThread, GROUP_AFFINITY GroupAffinity)
{
	GROUP_AFFINITY newGroupAffinity;

	newGroupAffinity.Group = GroupAffinity.Group;
	newGroupAffinity.Mask = GroupAffinity.Mask;
	
	if (SetThreadGroupAffinity(hThread, &newGroupAffinity, NULL) == 0)
	{
		//printf("\r\ndebug1:: hThread: %X, Group: %X, Mask: %X\r\n",hThread, newGroupAffinity.Group, newGroupAffinity.Mask);
		return -1;
	}
	else
	{
		return _getThreadNumber(GroupAffinity);
	}
}

Could you give more explanation on it? I'm still studying it, It's difficult for me to understand the sample.

Answer 6

A colleague of mine found the solution to this. When you call SetThreadGroupAffinity, bits in the reserved field of the GROUP_AFFINITY structure are being set (by the OS?). So, if the structure is reused after changing just the group and mask fields, the reserved bits are still set, causing the invalid parameter error.

To get around this you must zero out the entire structure before using it again.

ZeroMemory( &newGroupAffinity, sizeof(GROUP_AFFINITY));

-Tom

Answer 7

Yes 

ZeroMemory( &newGroupAffinity, sizeof(GROUP_AFFINITY));
works perfectly as your colleague suggest. Thank him/her for me. Will you?