Skip to main content

 none
RSS and multiprocessor groups RRS feed

  • Question

  • Hi there,

    I'm trying to understand RSS base processor allocation within specific processor group.

    Let's assume that I have 16 CPUs available.

    First case:

    If I set set-netadapterrss <Adapter_Name> -BaseProcessorNumber 20

    value 20 will be decreased to 16 (0:16) and this is quite simple and expected.

    Second case:

    bcdedit.exe /set groupsize 4

    Now I have 4 groups with 4 CPUs (0:{0,1,2,3}; 1:{0,1,2,3} etc)

    If I set set-netadapterrss <Adapter_Name> -BaseProcessorNumber 4

    System will assign BaseProcessor to 0:4 but it looks that this is not correct (I don't have processor of index 4 in 0 group)

    NOTE: https://docs.microsoft.com/en-us/windows-hardware/drivers/ddi/content/ndis/nf-ndis-ndisgetprocessorinformation will also return 0:4

    So back to the question, is the second case correct? Shouldn't value be decreased to highest processor from index 0 (0,3) group or be moved to first processor of the second group (1,0).

    Forgive me if this is wrong place to ask such questions but I can't find any information about Group:Processor base CPU assignment (https://docs.microsoft.com/en-us/windows-hardware/drivers/network/setting-the-number-of-rss-processors doesn't mention it at all)

    Regards,

    Robert

    Tuesday, October 16, 2018 12:33 PM

Answers

  • Hello,

    The value `16’ or `4’ is not a processor index so it doesn’t count from 0 it is actually a bit to turn on starting from bit position 1 within the group.  When you specify 0:16 it is represented internally as a set of bits that are turned on and in this case bit 16 is processor #15 if you numbered your processors starting from #0.  The representation of a set of bits is used for other purposes (e.g. affinity) but in this case we’re only turning a single bit on within a group.

     

    Similarly if you create 4 groups from your 16 processors and specify 0:4 then bit 4 is a valid processor in group 0.  If you choose to think of it in terms of an index starting from 0 it is really processor index #3 in group #0.

     

    If you think about it this is why groups are limited to 64 processors on a 64-bit architecture.  Below is a basic example of what happens internally but since you’re being asked to specify only one processor instead of a set we accept the number as some narrow integer that then expresses which bit to turn on.

     

     

    Bit Position      64 ... 8 7 6 5 4 3 2 1 (numbering starting from 1)

    e.g.                     0 0 0 0 1 0 0 0 <- “Processor Number 4 in whatever group this is”

    Obviously                0 0 0 0 0 0 0 0 <- “No processor within group, which is why “0” is not a valid processor”

     

    Anyway, if you specify something like 0:4 and 1:1 you’ve defined the last processor of group 0 and the first processor of group 1.

    Monday, October 22, 2018 6:40 PM
    Moderator
  • Hi Robert,

     

    I've taken a closer look at how these cmdlet's are interacting with both NDIS and the registry and unfortunately it appears to be somewhat asymmetric between set and get depending on the adapter presence & state.

     

    What I found was that the underlying WMI methods attempt to work with the adapter enabled or disabled.  Or in other words the methods attempt to continue working even if the resources being configured are not visible (e.g. processors in virtualized environments) or not enabled (such as the network adapter).

     

    For set, there is no attempt to apply any kind of validation to the values you provide and just stores them to the registry.  If the adapter is enabled it is restarted and NDIS internally reads, validates (and in the case your observing) adjusts the values for use by the adapter.

     

    For get, if the adapter is enabled the values in use by the adapter are presented in the output, otherwise the values from the registry are instead shown.

     

    Since you are probably running the cmdlet while the adapter is enabled this explains why what you set does not match your corresponding get.  It also means the only way to figure out what is being used is to run the get with the adapter enabled after the set and make sure it matches what you set.

     

    Another side-effect of that I discovered is you can end up running set commands that store nothing to the registry and mistakenly interpret the output of get as confirmation that the set was actually successful when in fact nothing has been stored and internally generated defaults are returned.  You can observe this by running get-netadapterrss on both an enabled and disabled adapter after running your set with out of range parameters.

     

    Finally, don't take my explanation of what is going on as this being ideal or correct behavior it just is how it works right now.  There is unfortunately limited latitude in adjusting the current behavior of the commands since it is likely to break compatibility in existing scripts if it were to be changed to be more intuitive.

     

    I hope this explanation helps.

     

    • Proposed as answer by tyler__ Wednesday, October 24, 2018 8:55 PM
    • Marked as answer by Robert Malz Thursday, October 25, 2018 11:09 AM
    Wednesday, October 24, 2018 5:29 PM

All replies

  • Any thoughts on this issue?
    Monday, October 22, 2018 11:09 AM
  • Hello,

    The value `16’ or `4’ is not a processor index so it doesn’t count from 0 it is actually a bit to turn on starting from bit position 1 within the group.  When you specify 0:16 it is represented internally as a set of bits that are turned on and in this case bit 16 is processor #15 if you numbered your processors starting from #0.  The representation of a set of bits is used for other purposes (e.g. affinity) but in this case we’re only turning a single bit on within a group.

     

    Similarly if you create 4 groups from your 16 processors and specify 0:4 then bit 4 is a valid processor in group 0.  If you choose to think of it in terms of an index starting from 0 it is really processor index #3 in group #0.

     

    If you think about it this is why groups are limited to 64 processors on a 64-bit architecture.  Below is a basic example of what happens internally but since you’re being asked to specify only one processor instead of a set we accept the number as some narrow integer that then expresses which bit to turn on.

     

     

    Bit Position      64 ... 8 7 6 5 4 3 2 1 (numbering starting from 1)

    e.g.                     0 0 0 0 1 0 0 0 <- “Processor Number 4 in whatever group this is”

    Obviously                0 0 0 0 0 0 0 0 <- “No processor within group, which is why “0” is not a valid processor”

     

    Anyway, if you specify something like 0:4 and 1:1 you’ve defined the last processor of group 0 and the first processor of group 1.

    Monday, October 22, 2018 6:40 PM
    Moderator
  • Thanks Cymon for detailed answer.

    To follow up on this issue.

    You stated that:

    The value `16’ or `4’ is not a processor index so it doesn’t count from 0 it is actually a bit to turn on starting from bit position 1 within the group.

    So what if we set: "set-NetAdapterRss <adapter> -BaseProcessorNumber 0"

    If in this function we specify bit (starting from index 1) what exactly 0 means?

    Also if I create CPU groups of size 4 and assign -BaseProcessorNumber to 5 or even higher number, the same value will be assigned to BaseProcessor

    Set-NetAdapterRss <adapter> -BaseProcessorNumber n

    Get-NetAdapterRss <adapter>

    BaseProcessor: [Group:Number]                   : 0:n
    Where < 64 

    First observation: I cannot assign 64 (but I can assign 0) so it could means that you might be wrong with the statement that we provide bit position and not processor index.

    Second observation: If my number of CPUs < 63 I can still assign n > number of CPUs, which was not possible when I had no groups (value is not "floored" down to my number of CPUs).

    note: tested on 17763 build

    So right now it looks like:

    • In set-NetAdapterRss we provide processor index (which is probably later translated to bit position),
    • User input is not validated by the OS/NDIS/Miniport*,
    • User input is validated in case there is no processor groups created (value is "floored" to number of CPUs)

    That leaves the questions:

    • Should BaseProcessorNumber value be validated or the OS trusts that I know what I'm doing?
    • * - Which component may be lacking of "group awareness"?

    • Edited by Robert Malz Tuesday, October 23, 2018 11:28 AM formating
    Tuesday, October 23, 2018 11:27 AM
  • Hi Robert,

    First, be aware I provided the first reply (which was kindly relayed by Cymon since I had no access to the forum at the time).

    Your follow-up (and observations) appear to be correct, but I would like to take the time to research the values provided through the (WMI) cmdlet and how they are used internally by both NDIS and miniports properly before responding.

    Thanks

    Tuesday, October 23, 2018 4:27 PM
  • Hi Robert,

     

    I've taken a closer look at how these cmdlet's are interacting with both NDIS and the registry and unfortunately it appears to be somewhat asymmetric between set and get depending on the adapter presence & state.

     

    What I found was that the underlying WMI methods attempt to work with the adapter enabled or disabled.  Or in other words the methods attempt to continue working even if the resources being configured are not visible (e.g. processors in virtualized environments) or not enabled (such as the network adapter).

     

    For set, there is no attempt to apply any kind of validation to the values you provide and just stores them to the registry.  If the adapter is enabled it is restarted and NDIS internally reads, validates (and in the case your observing) adjusts the values for use by the adapter.

     

    For get, if the adapter is enabled the values in use by the adapter are presented in the output, otherwise the values from the registry are instead shown.

     

    Since you are probably running the cmdlet while the adapter is enabled this explains why what you set does not match your corresponding get.  It also means the only way to figure out what is being used is to run the get with the adapter enabled after the set and make sure it matches what you set.

     

    Another side-effect of that I discovered is you can end up running set commands that store nothing to the registry and mistakenly interpret the output of get as confirmation that the set was actually successful when in fact nothing has been stored and internally generated defaults are returned.  You can observe this by running get-netadapterrss on both an enabled and disabled adapter after running your set with out of range parameters.

     

    Finally, don't take my explanation of what is going on as this being ideal or correct behavior it just is how it works right now.  There is unfortunately limited latitude in adjusting the current behavior of the commands since it is likely to break compatibility in existing scripts if it were to be changed to be more intuitive.

     

    I hope this explanation helps.

     

    • Proposed as answer by tyler__ Wednesday, October 24, 2018 8:55 PM
    • Marked as answer by Robert Malz Thursday, October 25, 2018 11:09 AM
    Wednesday, October 24, 2018 5:29 PM
  • Hi Tyler,

    Well, that replay actually answers a lot, I do really appreciate that.

    Of course new questions were born but I'll try to answer them myself.

    By the way, is there any reference where small fry like me can read more about these and other cmdlets?

    (more than generic information like https://docs.microsoft.com/en-us/powershell/module/netadapter/get-netadapterrss?view=win10-ps)

    Thanks,

    Robert

    Thursday, October 25, 2018 11:08 AM
  • Hi Robert,

    Unfortunately I had to learn the above by trolling through the source to understand what it was doing so there probably isn't any detailed documentation available beyond the link you referenced.

    I think the vision with the new docs.microsoft.com pages is that you can submit feedback on a topic (there's a panel that appears at the bottom of the page).  As I understand it real people do pay attention when pages start to get feedback that usually triggers questions to engineers internally to make these kind of investigations or clarifications.

    Otherwise open a post on the forum by topic and if the question is clear (as yours was) I'm sure the Microsoft engineers who monitor the forum will do their best.

    Thanks!

    Friday, October 26, 2018 3:44 PM