none
Interrupt Affinity with SMP RRS feed

  • Question

  • How are interrupts assigned to cores when using a multicore processor and SMP turned on? I read one of Doug Boling's posts that said that all the interrupts are handled by the main core when using SMP for a multicore processor. However, when I'm looking at Kernel Tracker with my quad core x86 (Baytrail) processor I'm seeing the same interrupts appearing at the same time on cores 1 and 3 (sometimes). How would one assign interrupts to a certain core? Is this possible? Trying to isolate certain interrupts to a certain core if possible.
    Sunday, November 29, 2015 2:24 AM

Answers

  • You have total control over your system, so you can pretty much do whatever you want. There is absolutely nothing stopping you from routing certain interrupts to a specific core and handling those ones on that core. I'm not sure if you actually can on x86 (I'm more of an ARM expert, and on ARM we can route specific interrupts to specific cores if we would want).

    However, CE is an SMP system, meaning the system itself determines on which core to run a thread (it load-balances itself), so even if you are going to send a specific interrupt to a specific core, there is no way you can stop the CE kernel from loading up that particular core, and this could influence your response times (depending on the thread priority of the threads running on that core).

    Setting thread affinity is great, but it only affects YOUR threads (none of the other system threads take any notice of that), and it usually results in worse behavior for your application. In almost all cases it is much better to let the system determine the best core to run your thread on.

    There is of course another way and that is to exclude your "real-time" core from the SMP cluster. That way you are in TOTAL control of any code that core runs...

    A lot more work, for sure, but depending on your application this may be the best way to go (but probably not).



    Good luck,

    Michel Verhagen, eMVP
    Check out my blog: http://guruce.com/blog

    GuruCE
    Microsoft Embedded Partner
    http://guruce.com
    Consultancy, training and development services.

    • Marked as answer by elk84 Thursday, December 3, 2015 2:54 AM
    Wednesday, December 2, 2015 11:11 PM
    Moderator

All replies

  • Hi Elk,

    My understanding is that all interrupts are all handled by the primary core as discussed in this article. http://www.rtcmagazine.com/articles/view/101957. However, it is up to the BSP manufacturer to specify that core and potentially there is something being done at the BSP level on your board that is changing that behavior? Have you reached out to your BSP Provider? I would also be very interested in this answer as I have not seen the behavior you indicate on my dual core CEPC but it uses the built in CEPC BSP.

    Sincerely,

    IoTGirl

    Monday, November 30, 2015 6:27 PM
    Moderator
  • Hi IoTGirl,

    Our BSP provider isn't of much help, its basically the generic BSP from Intel, I will probably investigate this myself. With your dual core CEPC do you only see SYSINTR #0 or SYSINTR #1 on a single core, or is it on both cores when viewing activity using kernel tracker? It's possible that the system tick is duplicated on all the cores, but not all interrupts are.

    How is it possible to utilize certain cores for RT processing if all the system wide interrupts only get directed to a single core? It would be ideal to have an RT core isolated that doesn't see any miscellaneous interrupts (if there was a way to mask the interrupts for certain cores), and only see the pertinent interrupts meant for real time processing. I'd like to have one of the cores handling non-rt interrupts (i.e from the keyboard, usb ...) and a separate core that handles interrupts that deal with real time processing. I think I'm missing something basic here with respect to how interrupts deal with a multicore processor. 

    Thanks

    Wednesday, December 2, 2015 5:19 PM
  • Hi Elk84,

    You can set thread affinity and control which core is being used per-process but doug clearly states "New API calls have been added to manage the multiple cores. Applications can set a thread’s affinity to a specific core, or all threads to a specific core. In addition, developers can power down cores (with the exception of the primary core) of the CPU to save power. All interrupts are handled by the primary core of the CPU."

    For a picture view, Glen was the BSP PM for Compact 7 and produced the following white Paper that might help in your quest for basic knowledge. http://download.microsoft.com/download/2/4/A/24A36661-A629-4CE6-A615-6B2910A1367A/Symmetric%20Multiprocessing%20Guide%20for%20Windows%20Embedded%20Compact%207.pdf After reviewing this doc, I would recommend comparing your BSP with the CEPC BSP that ships with Compact.

    Sincerely,

    IoTGirl

    Wednesday, December 2, 2015 10:54 PM
    Moderator
  • You have total control over your system, so you can pretty much do whatever you want. There is absolutely nothing stopping you from routing certain interrupts to a specific core and handling those ones on that core. I'm not sure if you actually can on x86 (I'm more of an ARM expert, and on ARM we can route specific interrupts to specific cores if we would want).

    However, CE is an SMP system, meaning the system itself determines on which core to run a thread (it load-balances itself), so even if you are going to send a specific interrupt to a specific core, there is no way you can stop the CE kernel from loading up that particular core, and this could influence your response times (depending on the thread priority of the threads running on that core).

    Setting thread affinity is great, but it only affects YOUR threads (none of the other system threads take any notice of that), and it usually results in worse behavior for your application. In almost all cases it is much better to let the system determine the best core to run your thread on.

    There is of course another way and that is to exclude your "real-time" core from the SMP cluster. That way you are in TOTAL control of any code that core runs...

    A lot more work, for sure, but depending on your application this may be the best way to go (but probably not).



    Good luck,

    Michel Verhagen, eMVP
    Check out my blog: http://guruce.com/blog

    GuruCE
    Microsoft Embedded Partner
    http://guruce.com
    Consultancy, training and development services.

    • Marked as answer by elk84 Thursday, December 3, 2015 2:54 AM
    Wednesday, December 2, 2015 11:11 PM
    Moderator
  • Hi IoTGirl,

    Thanks for the documentation, I will compare my BSP to the CEPC bsp and follow up.

    Thanks

    Wednesday, December 2, 2015 11:28 PM
  • Hi Michel,

    Thank you for your response, I think you made this more clear for me now. So in order to route interrupts to a certain core I would assume this is done at the OAL level, perhaps in fwpc.c? Even on ARM, where would one look to see where interrupts are being routed?  

    When you mention system threads I am thinking about the miscellaneous threads or processes I see running in kernel tracker such as drivers like gwes.dll or processes like explorer.exe or any of the other drivers that are loaded. Are you referring to driver threads such as these, or other system threads that will not show up in kernel tracker? The reason I ask is, is it not possible to take one of my own threads and essentially assign it to a core (using threadaffinity), set the threadquantum to zero, set the threadpriority high (really high priority) and let it essentially consume all the time of the core not allowing any other threads (even system threads) from running on that particular core? Or are there certain system threads that will run no matter what and take some of that core's processing time? Would those threads show in kernel tracker (assuming it's not one of the dlls or processes I already see? 

    Thanks,

    Wednesday, December 2, 2015 11:50 PM
  • So in order to route interrupts to a certain core I would assume this is done at the OAL level, perhaps in fwpc.c? Even on ARM, where would one look to see where interrupts are being routed?

    In our BSP we set all of this stuff up very early in the bootprocess (in our case the file is cestartup.s). In x86 it will have to be done early in the CE bootprocess as well, but it's been too long to remember exactly where this is done. I don't think it will be in fwpc.c though... IIRC that only contains the main ISR code.

    When you mention system threads I am thinking about the miscellaneous threads or processes I see running in kernel tracker such as drivers like gwes.dll or processes like explorer.exe or any of the other drivers that are loaded. Are you referring to driver threads such as these, or other system threads that will not show up in kernel tracker?

    Any thread, no matter if it is a 'system' thread, driver thread or application thread will show up in kernel tracker (there are no 'hidden' threads), and there is really no distinction either.

    The reason I ask is, is it not possible to take one of my own threads and essentially assign it to a core (using threadaffinity), set the threadquantum to zero, set the threadpriority high (really high priority) and let it essentially consume all the time of the core not allowing any other threads (even system threads) from running on that particular core?

    That sounds like a tremendously bad idea. How will you select a core? How are you going to be sure there are no threads running on that core to begin with? You are going to starve those threads already running on that core and in fact you will kill the entire system with that approach.

    I think first of all, before even thinking about these things, you will have to determine if the real-time performance is not enough for your application. You will most likely find it is good enough and you don't need to resort to any special code at all.


    Good luck,

    Michel Verhagen, eMVP
    Check out my blog: http://guruce.com/blog

    GuruCE
    Microsoft Embedded Partner
    http://guruce.com
    Consultancy, training and development services.

    Thursday, December 3, 2015 2:40 AM
    Moderator
  • Ok great, thanks for the advice Michel!
    Thursday, December 3, 2015 2:59 AM