working set, virtual bytes and private bytes

Question

Hello everyone,

 

I am using Windows Server 2003 Performance Counter tool to monitor the memory consumed by my process.

The interested terms are working set, virtual bytes and private bytes. My questions are,

 

1. If I want to watch the real physical memory consumed by current process, which one should I monitor?

2. If I want to watch the physical memory + swap file consumed by current process, which one should I monitor?

3. Any more clear description of what these terms mean? I read the help from Performance Counter tool, but still confused

which one(s) identifies the real used physical memory, and which one(s) identifies the real used physical memory + page swap file, and which one(s) identifies the required memory (may not be really allocated either in physical memory or in swap page file).

 

If there are any related learning resource about the concepts, it is appreciated if you could recommend some. :-)

 

thanks in advance,
George

Answer 1

Your question is not specific to the C++ language so I suggest you move to the SDK forum instead.

To monitor memory usage, use performance counter instead of task manager.

You can find the terminology materials in the Technet's Performance Counters Reference pages.

Answer 2

Thanks Sheng Jiang,

 

 

I will re-post there.

 

Sheng Jiang (蒋晟) wrote:

Your question is not specific to the C++ language so I suggest you move to the SDK forum instead. To monitor memory usage, use performance counter instead of task manager. You can find the terminology materials in the Technet's Performance Counters Reference pages.

 

 

regards,

George

Answer 3

The physical memory usage can not be determined, all you can see is how much memory currently your program is using, that memory means the allocated memory, there is no good way to separate the swap and physical RAM from it.

From MSDN: The working set of a program is a collection of those pages in its virtual address space that have been recently referenced. It includes both shared and private data. The shared data includes pages that contain all instructions your application executes, including those in your DLLs and the system DLLs. As the working set size increases, memory demand increases.

And as I understand, the Virtual bytes are bytes allocated in virtual memory (using VirtualAlloc etc) and private bytes are bytes allocated in local heap (using malloc, new etc).

From MSDN: Private bytes include memory that is committed and marked MEM_PRIVATE, data that is not mapped, and executable pages that have been written to.

Answer 4

Thanks Ramkrishna,

 

 

From your reply, I have two more comments, I am not sure whether you agree or not?

 

1. Working Set only refers to physical RAM consumed (not including page swap file);

2. Private byte does not include the virtual memory, which is reserved, but not committed.

 

Are (1) and (2) correct?

 

Ramkrishna Pawar wrote:

The physical memory usage can not be determined, all you can see is how much memory currently your program is using, that memory means the allocated memory, there is no good way to separate the swap and physical RAM from it. From MSDN: The working set of a program is a collection of those pages in its virtual address space that have been recently referenced. It includes both shared and private data. The shared data includes pages that contain all instructions your application executes, including those in your DLLs and the system DLLs. As the working set size increases, memory demand increases. And as I understand, the Virtual bytes are bytes allocated in virtual memory (using VirtualAlloc etc) and private bytes are bytes allocated in local heap (using malloc, new etc). From MSDN: Private bytes include memory that is committed and marked MEM_PRIVATE, data that is not mapped, and executable pages that have been written to.

 

 

regards,

George

Answer 5

No, the working set does not refer to physical memory, it is the program memory from virtual address space.

Answer 6

Then, Ramkrishna.

 

 

In your points, there is no differences between the amount of virtual memory and the amount of working set, right? It is not true.

 

You can open your perfmon and check the data for virtual memory and working set.

 

Ramkrishna Pawar wrote:

No, the working set does not refer to physical memory, it is the program memory from virtual address space.

 

 

regards,

George

Answer 7

No, see, the memory used by application's code loading, DLLs in address space etc. comes under Working set.

The virtual memory is something the application allocated after starting. I am not sure how the addition and calculation of these params would be done from other raw parameters.

Answer 8

Hi Ramkrishna,

 

 

Here is Microsoft formal definition of working set.

 

http://support.microsoft.com/kb/108449/en-us

 

A process's working set is the set of pages that it has currently in memory.

 

So, I think working set is the amount of memory (physical, RAM, not page swap file) consumed by current process. I am not an expert of this topic. If I am wrong, please feel free to correct me. :-)

 

Ramkrishna Pawar wrote:

No, see, the memory used by application's code loading, DLLs in address space etc. comes under Working set. The virtual memory is something the application allocated after starting. I am not sure how the addition and calculation of these params would be done from other raw parameters.

 

 

regards,

George

Answer 9

Yes, the MSDN defination is not diff from what I have posted here. The pages in memory are all virtual (not physical memory) these pages are loaded into physical RAM when the process get's it's turn to run code on processor, when the process is not active (when processor is executing other process's code, according to timeslots) then these pages (or the contents of those pages) are pushed into swap (or in other words the hard disk file for paging virtual memory).

You should read about the Windows Memory Management to get more insight into this.

Answer 10

Thanks Ramkrishna,

 

 

I am wondering what do you think the amount of working set and the amount of committed + reserved virtual memory of current process? Are they the same?

 

I think you think they are the same. But my point is, only the virtual memory loaded into RAM could be part of working set, and the amount of part of virtual memory which is in swap file is not part of working set. I am not sure whether you agree with me or not? :-)

 

Ramkrishna Pawar wrote:

Yes, the MSDN defination is not diff from what I have posted here. The pages in memory are all virtual (not physical memory) these pages are loaded into physical RAM when the process get's it's turn to run code on processor, when the process is not active (when processor is executing other process's code, according to timeslots) then these pages (or the contents of those pages) are pushed into swap (or in other words the hard disk file for paging virtual memory). You should read about the Windows Memory Management to get more insight into this.

 

 

regards,

George

Answer 11

For the benefit of people that stumble across this page in the future:


Working set is the subset of virtual pages that are resident in physical memory only ; this will be a partial amount of pages from that process.

http://blogs.msdn.com/salvapatuel/archive/2007/10/13/memory-working-set-explored.aspx