locked
Process ID & name - is this how NM obtains it? RRS feed

  • Question

  • Hi,

    I'm curious to know how Network Monitor is getting the process id/process names.  Would this be the way it is done?

    * From the packet captured, get the source & destination IP address and TCP ports

    * Iterate through the rows from IP Helper "GetTcpTable" until you find a match for all 4 (four) source & destination IP address and ports numbers

    Is this the most robust way to do the correlation?   Also I assume this means you'd have to be doing the correlation in real time to ensure the TcpTable entry didn't drop away? 

     

    thanks

    (an extension of the discussion here that I though should be in it's own thread - http://social.technet.microsoft.com/Forums/en/netmon/thread/73a62670-264a-404c-bde3-c78121e2c6a3

     

     

    Friday, July 16, 2010 5:31 AM

Answers

  • Yes, that is the basic algorithm.  We take the source/dest IP address and TCP Ports and then match it to an entry in the table. Once we get a match, we query the OS using the process ID and can get things like the path and process name so that we can store the ICON if one exists.

    If I've misunderstood your question, please let me know.

    Paul

    • Proposed as answer by Paul E Long Wednesday, July 21, 2010 4:24 PM
    • Marked as answer by callagga Wednesday, July 21, 2010 8:53 PM
    Wednesday, July 21, 2010 4:24 PM
  • Hello,

    The rate at which NM queries the TCP/UPD connection table depends on 3 factors:

    1. The rate of incoming frames.  The NM driver sends a buffer load of frames to the NM process at least once per second.  If the buffers fill faster than that, it sends them as soon as they fill up.  The default buffer size is 512 MB (~500 frames).  Upon receiving a buffer, the NM process will take a snapshot of the TCP/UDP connections for IPV4 and IPV6.
    2. The interval indicated by the user's preference.  Using a tool like regedit, a user can create a DWORD value at HKCU\Software\Microsoft\Netmon3\Settings\ProcessTracking, ProcessUpdateInterval.  This value is the time period in milliseconds to wait between querying.
    3. The NM driver buffer state.  If the NM process begins to fall too far behind the NM driver, NM will stop querying process information until it catches up.  Since we query in the 'hot' path and querying can be expensive, this provides a great savings to prevent dropping frames.  In other words, we sacrifice process accuracy to collect as many frames as possible.

    For expiration, we default to 10 minutes.  In the same registry location indicated above, the user can create a DWORD value, ProcessExpiration, which specifies the time limit, in minutes, for keeping connection information. 

    As you correctly assumed, we use IPHelper API to collect the connection information which includes process IDs.  We are at the mercy of this API for how accurately we correlate the frames to processes, assuming we take a snapshot in time to see the connection.  There are a few situations where frames are not associated with the process which sends and receives the network data.  For example, this can occur when a service acts as a broker between the process and the network.  In these cases, the traffic will be associated with svchost.exe instead of the real process which is requesting network data.

    Hope this helps.

    Darren Fisher - Network Monitor Development Lead

    Monday, July 26, 2010 5:44 PM

All replies

  • Hello again,

    We poll the table at set intervals and update our own copies of the process table.  There's a chance that we may miss short-lived processes, but we can't continually do the polling as it's an expensive operation.

    From there we use other various APIs found on MSDN to retrieve the process handle, get the filename http://msdn.microsoft.com/en-us/library/ms683198(v=VS.85).aspx, and in the UI case get the icon.


    Michael Hawker | Program Manager | Network Monitor
    Friday, July 16, 2010 9:27 PM
  • thanks Michael - the bit particular bit I'm keen to understanding however is the mapping from the captured packets across to the process info from the windows API.  My assumption is that the capture capture doesn't even give you a process id for example.  To the matching/identification of what row in the TCP table the packet aligns with are you doing what I was guessing at can I ask?  That is:

    * From the packet captured, get the source & destination IP address and TCP ports

    * Iterate through the rows from IP Helper "GetTcpTable" until you find a match for all 4 (four) source & destination IP address and ports numbers

    thanks

    Friday, July 16, 2010 10:12 PM
  • Yes, that is the basic algorithm.  We take the source/dest IP address and TCP Ports and then match it to an entry in the table. Once we get a match, we query the OS using the process ID and can get things like the path and process name so that we can store the ICON if one exists.

    If I've misunderstood your question, please let me know.

    Paul

    • Proposed as answer by Paul E Long Wednesday, July 21, 2010 4:24 PM
    • Marked as answer by callagga Wednesday, July 21, 2010 8:53 PM
    Wednesday, July 21, 2010 4:24 PM
  • yes that's exactly what I was curious about - thanks

    don't worry about answering the following unless you happen to know offhand but often does NM poll the TCP table & how long does it retain entires for after not being used (noting you mentioned you don't grab TCP details whilst ever packet is coming through) - I'm thinking that allow NM to display a process ID relies on the fact that the time between the packet coming in, and when NM next polls the TCP table, the details don't drop out of the TCP table?   This would mean also the display of the packet in the UI (which includes process names) would have to wait for the next TCP table poll too, so I'm guessing you must poll a few times a second perhaps?

     

     

    Wednesday, July 21, 2010 9:01 PM
  • Hello,

    The rate at which NM queries the TCP/UPD connection table depends on 3 factors:

    1. The rate of incoming frames.  The NM driver sends a buffer load of frames to the NM process at least once per second.  If the buffers fill faster than that, it sends them as soon as they fill up.  The default buffer size is 512 MB (~500 frames).  Upon receiving a buffer, the NM process will take a snapshot of the TCP/UDP connections for IPV4 and IPV6.
    2. The interval indicated by the user's preference.  Using a tool like regedit, a user can create a DWORD value at HKCU\Software\Microsoft\Netmon3\Settings\ProcessTracking, ProcessUpdateInterval.  This value is the time period in milliseconds to wait between querying.
    3. The NM driver buffer state.  If the NM process begins to fall too far behind the NM driver, NM will stop querying process information until it catches up.  Since we query in the 'hot' path and querying can be expensive, this provides a great savings to prevent dropping frames.  In other words, we sacrifice process accuracy to collect as many frames as possible.

    For expiration, we default to 10 minutes.  In the same registry location indicated above, the user can create a DWORD value, ProcessExpiration, which specifies the time limit, in minutes, for keeping connection information. 

    As you correctly assumed, we use IPHelper API to collect the connection information which includes process IDs.  We are at the mercy of this API for how accurately we correlate the frames to processes, assuming we take a snapshot in time to see the connection.  There are a few situations where frames are not associated with the process which sends and receives the network data.  For example, this can occur when a service acts as a broker between the process and the network.  In these cases, the traffic will be associated with svchost.exe instead of the real process which is requesting network data.

    Hope this helps.

    Darren Fisher - Network Monitor Development Lead

    Monday, July 26, 2010 5:44 PM
  • Hey Darren - was just curious - if you gather up a bunch of packets for processing (say 1 seconds worth) and then you  poll the TCP Table for it's data, is there a reason why you have to maintain TCP entries for a given period (your expiration 10 minutes)? 

    Would it be to cover the situation that during a 1 second period if the connection is closed it drops out of the TCP Table immediately hence you need to keep some history?

     

     

    Thursday, August 12, 2010 2:45 AM
  • It's because we're buffering the data for parsing.  Since we may not be able to keep up with the network traffic, we place all the incoming frames in a disk buffer (in the default case).  This is the same time we poll the table and recover the TCP/UDP connection info.

    However, we only associate that data to the frame when we parse it which could be much later since the buffer could be hundreds or thousands of frames.  Thus, we keep the table around in our own memory space in case the process disappears in that time frame before we pass it to ensure we can still associate the data with the frame when we get to looking at it.


    Michael Hawker | Program Manager | Network Monitor
    Thursday, August 12, 2010 8:43 PM
  • Got it.  Out of curiosity Michael did you notice how long Windows keeps the connections listed in the TCP Table?  Or does it remove them once a connection is closed (I'm kind of guessing here TCP connections are explicitly closed)
    Thursday, August 12, 2010 9:06 PM
  • Michael or Paul - I've noted sometimes NM doesn't provide a process name.  The column may be blank sometimes.   Do you understand why this is the case?  Is the process of getting process names by the mapping of the source/dest IP/port combinations to the Windows TCP table not working in all cases?

     

    thanks

    Monday, October 4, 2010 8:00 AM
  • There can be cases where short lived processes are gone by the time we check the TCP connection.  Network/CPU load can affect this as well.  If we determine we are too busy and will drop frames, we can also turn of process tracking.  And obviously broadcast traffic would not have a process associated with it.

    Paul

    Monday, October 4, 2010 4:36 PM