Performance: Memory

Introduction to Memory Bottlenecks

If ever your Windows 2003 server is running slowly, then the first place to look for a bottleneck is memory. Another way of looking at server performance, is that machines with plenty of RAM rarely give problems. A bonus of plenty of memory is, that to a degree, abundant RAM compensates for strain on other resources.

On old servers, lack of memory would give you the full sensory input, you could hear the disk paging, see the light flashing, and 'Mad' Mick swears you could smell the disk thrashing. Even with these sensory clues, it is still worth while monitoring memory with Performance Logs. Please also remember the big picture. So once you have had a quick look at memory, remember to check the processor and disk counters.

The servers most likely to suffer from memory shortage are pure database servers for example, Oracle or SQL. Email server also require plenty of RAM. Pure domain controllers are less likely to experience memory problems.
Memory Topics

* Available Bytes
* Three Paging counters
* Other Memory Counters
* Identify the root cause of a memory bottleneck
* Advice for application servers e.g. Exchange, SQL or Web
* Solutions to memory problems

Key memory counter: Available Bytes

The more available memory the faster the server can respond. When I check a server's memory with performance monitor, the first counter that I add to the log is Memory\Available bytes. As long as the trace indicates more than 10MB of free memory, then I conclude that the server has sufficient RAM.

Diagram 1 shows a white descending line, and the legend confirms that Available bytes are down to 3MB. Clearly this machine needs more memory.Performance Monitor, Memory bottleneck, Available Bytes

Suppose a spreadsheet wants to start a new thread or a database needs to sort data, what each needs is memory. The operating system provides this memory at least 100 times faster using RAM, than it could using a disk based pagefile. This is why a large pool of free memory is so important to an application server.
Three paging counters in performance monitor
1) Memory: Pages / sec

Take care to distinguish between these two paging counters:
1) Pages /sec (Hard page faults)
2) Page Faults /sec counter is likely to be at least twice the value of the above.

Two problems with monitoring in general, firstly no counter should be taken in isolation, secondly spikes should be ignored, or at least played down.

The less paging the better your server's performance. Most authorities agree that Memory: Pages / sec is a key memory counter. This counter measures 'hard' page faults, in other words the page in nowhere in memory, so the VMM (Virtual Memory Manager) has to fetch the data from the pagefile on the disk; in computing terms that takes an age.

I am reluctant to disagree with other authorities, but from my experience, I would put the threshold as high as 20 pages /sec, before blaming paging as the bottleneck. Moreover, I would not trust pages /sec as an indicator of a bottleneck without confirmation from low Available bytes. (see above)

In truth, if you put 5 experts in the same room, they could all spot a memory bottleneck, but when they wrote up their notes, they would use different time slices and different thresholds, consequently, it would seem that there was a conflict where none actually existed.

2) Memory: Page Faults / Sec

Page faults / sec is the sum of hard and soft page faults. Soft page faults are where the data is found elsewhere in RAM. For example, Word has opened the spellchecker, and now Outlook wishes to use it, there is no need for another call to the disk as the spellchecker is already in memory.

Hard page faults are generated whenever the VMM has to fetch data from the pagefile on the disk.
3) Page File: % Usage

While the pagefile is less likely to be a bottleneck it is easy to check, and satisfying to fix. You could also confirm that it is on the most suitable disk, and if possible split the pagefile over two disks and thus improve access times. (Note the object here is Page File not Memory)

Because the changes are so gradual, you are better off using this Page File % Usage counter as an alert, rather than a log. I suggest setting an alert on a limit of over 70.

Creating a Memory Bottleneck

If you really want to see a memory bottleneck that you can measure with performance monitor, then add the MAXMEM switch to your server's boot.ini. For example: multi(0)disk(0)rdisk(0)partition(1)\Windows="Windows Server 2003" /MAXMEM=256
Other memory counters
For leaky Applications
Memory: Pool Non-paged Bytes

Thanks to better programming, the problem of applications leaking memory is slowly disappearing. Programs need memory - fair enough, but when they close, if they fail to release the memory back to the pool - not fair.

Monitoring the slow build-up in the non-paged pool is a classic job for an alert. You are unlikely notice much creep in a day's logging, but over a month you could have lost a significant amount of memory. If you are in a hurry or just plain ruthless, a reboot solves memory leakage into the non-paged pool.
Memory: Cache Bytes

With Memory: Cache Bytes, you have the choice of monitoring the total or breaking down the cache into four categories.

Note Memory: Cache Bytes = Total of these four individual counters

* Memory: System Cache Resident Bytes
* Memory: System Driver Resident Bytes
* Memory: System Code Resident Bytes
* Memory: Pool Paged Resident Bytes

Cache: Data Map Hits %

Data Map Hits is the percentage of data maps in the cache that could be resolved without having to call the disk. It is a measure of how many data pages are already in physical memory. (Note the object is Cache not Memory like the others.)
Committed Bytes

If the value for committed bytes is greater than physical memory, then more RAM would help.
Identify the root cause of a memory bottleneck

Your goal is to discover which application (process) is using most of the memory. Either quickly check the Processes tab of Task Manger or better, create a log of the processes with performance monitor. Developers could consider investigating other tools such as PViewer from the Resource Kit.
Advice for application servers e.g. Exchange, SQL or Web

Many applications have their own settings for configuring memory usage. It is not that they can magically produce more memory but they can make the most of the memory the server has. Programs like SQL or Exchange have settings to control their caches. The key question is: should the memory be tied up in a file cache or would it better used by the application's own cache? I favour the latter. So adjust the applications cache or memory settings, and then see what effect your changes have by logging with System Monitor.

See more about Exchange Memory here
Solutions to memory problems Memory Bottlenecks

Not enough available memory? The easiest cure for memory problems is to open up the server and add another stick of RAM. Salesmen always seem particularly helpful and knowledgeable on the best type of RAM for my server.

I have a suggestion for future purchases - Always buy machines with more RAM than you need now.
More help for detecting bottlenecks