Interrupt Priority in Kernel Tracker RRS feed

  • Question

  • Hello,

    I am trying to understand how interrupt priority is set within Windows CE. Is the interrupt priority dealt with by the x86 processor and then the kernel creates a stack of interrupts to be serviced? I would like to make one of my interrupts a higher priority so it preempts any other interrupts that arrive. I see various nestlevels in kernel tracker, but I don't know what that exactly means. Here is my specific situation:

    I am using the local APIC to generate interrupts periodically, however I have noticed that when other interrupts occur close to the time of the LAPIC interrupt there is a slight delay for the LAPIC interrupt. I would like to increase the interrupt priority of the timer using the LAPIC to be higher than that of all the interrupts that are routed through the 8259 pic. The local apic can be programmed to interrupt any of the interrupt vectors of the processor, would this be the only way of setting the interrupt priority?

    When I'm looking at activity in kernel tracker I notice that the interrupts that are occuring on my x86 platform each have a certain nestlevel that ranges from 0 to 2. I am trying to understand how the interrupts are prioritized when using Windows CE with an x86 processor. From my understanding each interrupt is mapped to a vector number, and the lower vector numbers have priority over the higher ones. Now I'm trying to understand how the kernel interacts with the interrupts, and how it knows which interrupts are higher priority so that it can preempt lower priority interrupts. 

    Is there anything that can be done from within Windows CE to set the interrupt priority of certain interrupts, or is Windows CE just nesting the interrupts based on the order they are received from the processor? 

    Friday, January 8, 2016 8:05 PM

All replies

  • I am not an x86 expert, so I will leave the hardware to you and anyone else who may answer.

    But in general, Windows CE doesn't do anything - specifically, your BSP handles and prioritizes interrupts.  It can do this in two ways; first if the processor has hardware priorities, it can set them, second, how OEMInterruptHandler() processes the interrupts also can set a priority but only if the hardware presents multiple interrupts at one time (which would indicate that there aren't hardware priorities, or that they are being ignored).

    Bruce Eitman (eMVP)
    Senior Engineer
    Bruce.Eitman AT Eurotech DOT com
    My BLOG
    I work for Eurotech

    Friday, January 8, 2016 9:02 PM
  • I think the value of the vectors assigned to each interrupt determine the interrupt priority with x86 although I'm not sure. The CEPC equivalent of OEMInterruptHandler is PerPISR (I think), and this basically just queries the 8259 for which interrupt is present, then it returns the appropriate SYSINTR based on what the 8259 returns. I wrote a separate ISR that handles the LAPIC timer, it simply returns a SYSINTR. From kernel tracker I can definitely see that there are nest levels for the interrupts, however I don't think this has anything to do with the ISR. Does anyone know which code in the Private directory handles nesting interrupts, or is it usually somewhere in the OAL? I have also seen references to people turning off nested interrupts, I would like to know how the kernel knows the priority of the interrupts to allow for the nesting that I am seeing in Kernel Tracker. It would also be nice to be able to see all the interrupts that are registered with the kernel somehow.
    Saturday, January 9, 2016 1:42 AM
  • Check this

    It contains source code to operate in APIC mode. This code is used in production code with success.

    It uses the ACPICA library to query (the BIOS) for APIC, LAPIC, IOAPIC configuration of your (x86) hw, but as you already have modified some of the (cloned CEPC) bsp, I think you can figure out how things work yourself easily.

    Regarding your question.

    1. You can (re)program the IOAPIC to control the hardware IRQ priorities. This gives you control over how the hw schedules the IRQs. In Windows CE all hw IRQs are send through the vector table to PerPISR() ISR (interrupt service routine)

    2. Once a hw IRQ is received in the ISR, PerISR will dispatch it to a Windows CE IST (inrerrupt service thread). This way you can control the IST priority yourself.

    The idea in Windows CE is that you do minimal work in the ISR and dispatch asap to a "normal" thread (IST). This way you have thread priority control over your hw interrupts => more flexibel.


    Tuesday, January 12, 2016 8:30 AM
  • Hello Werner,

    I actually have been reading your blog on this topic in the past, I was just hoping that I would utilize the LAPIC without the IOAPIC (LAPIC generated interrupts directly to the processor while leaving the PIC to handle hardware interrupts), I didn't want to get into switching over how the interrupts are routed (although I regret my decision now and wish I just followed your blog's instructions). I believe in the future I will follow your blog's instructions, however now I am short on time.

    What I did was to create a second ISR in the OAL alongside of PerPISR(). Since PerPISR() just queries the pic for the interrupts, it has nothing to do with the interrupts routed directly to the processor from the APIC. I am able to specify the vector value that interrupts when the APIC counts down to zero. From my understanding the lower valued vectors would have priority, however I don't see much of a difference from choosing lower valued vector values for the interrupt performance wise. The slight issue I had was that when the timer would interrupt (say every 250 microseconds cyclically), every 75 ms I would see a 6 microsecond delay, which when looking at kernel tracker was other interrupts arriving at slightly before or at the same time as my APIC interrupt (primarily the 1 ms system tick with its slight drift). I believe PerPISR() can be modified somehow so that those interrupts can be temporarily masked off at the times my APIC timer is causing an interrupt, but I have not thought of a way to do this. 

    I believe using ACPICA may be the best way to go in the future, especially when I start adding more source of interrupts (in order to prioritize them). Have you tried using the APIC as a timer? Have you observed its consistency when running? Is there any interference from other interrupts as I have seen? Thanks for you work on the topic!

    Tuesday, January 12, 2016 9:46 PM
  • ACPICA is the longer (more difficult, but more complete) way to go if you wish to query hw specific details of your x86 motherboard, in this case the APIC and IOAPIC.

    If you wish to stick to PIC and you suspect priority issues, I would advise to remap the interrupt vector table in x86InitPICs(). Search for HookInterrupt(), with this function you have control over IRQ to ISR priority order.

    If you are looking at microseconds response/delays this priority order/vector table entry order is important. However INTR_TIMER0 is responsible for the OS (RTOS) internal (thread) scheduling, so you might not want to delay (lower priority) this one too much.

    Ultimately you might need to reprogram the PIC too to reorder its priorities... (APIC has more possibilties).

    But i would try to avoid this. The whole idea behind IRQ -> ISR (PerPSIR) -> IST is that you have interrupt priority control via IST. However if you really need (sub) microsecond accuracy, you might need to install your own ISR or priority order and handle everything from there.

    Good luck (how often a day you reboot/repower your device :-) )


    Wednesday, January 13, 2016 8:01 AM
  • Hi elk84,

    did you consider to use generic PIC mode, where HPET/IRQ12 routed to separate ISR? It allows to increase timer intr priority implicitly.
    I've implemented similar block for Timer0/IRQ0 on Baytrail boards due to nested interrupts issue.
    BTW, what is your hardware?

    Best regards, Igor

    Wednesday, January 13, 2016 9:28 AM
  • Hi Werner,

    It was actually after looking at this function that I realized that it may not be an interrupt priority issue, rather just the delay it takes for the OS to preempt the interrupts of lower priority. In x86InitPics() I have:

        // Setup the PeRP interrupt handler and enable the PeRP interrupt in the BasePSR 
        for (i = 64; i < 80; i++)
            HookInterrupt(i, (FARPROC)PeRPISR);

    This implies that they are mapping the pic IRQs to interrupt vectors 64->80. When I hooked my interrupt I used a lower vector value (and that is why I was confused with the performance I was seeing). The issue is that cyclically other interrupts (such as the system timer) would interrupt slightly before my APIC interrupt, so I'm guessing the OS was preempting PerPISR() with my ISR for the APIC, and this causes a delay. Worse yet, sometimes 3 different interrupt sources would occur at roughly the same time with my APIC interrupt being the last interrupt source to occur, and when this happens I would see the 6 us delay (which would be the time the OS is preempting the other 2 ISR's and servicing my ISR. I'm just speculating, but I think this is what is happening (and it's not an issue of interrupt priority). 

    Thanks for the insight, I learned a lot from your blog. My device should be able to go weeks without being rebooted/repowered (although I haven't tested this yet).

    Wednesday, January 13, 2016 5:29 PM
  • Hi Igor,

    I too am using the Baytrail platform, specifically the E3845. What has been your experience with the HPET an the Baytrail platform? I looked into using the HPET, actually from another one of Werner's blogs:

    However, he mentions that the HPET was buggy (and others have mentioned this). That is why I went the APIC route. What has been your experience with the HPET on Baytrail? Are you able to achieve periodic interrupts with microsecond accuracy? I would not mind looking into this method if you know it works!

    Wednesday, January 13, 2016 5:34 PM
  • Hi elk84,

      I did use HPET for experiments only, so I haven't live results.
    But I'm sure it's impossible to obtain microsecond's accuracy for IST, it's more feasible for ISR.
    Also check my experience with loop jitter -

    Best regards, Igor

    Friday, January 15, 2016 7:37 AM