We use four ESX-based Exchange 2010 SP3 UR5-v2 VMs (Multi Role, DAG) experiencing strong memory issues every 6 to 8 weeks. It starts with an increasing number of event ESE/906 "Information Store (4032) A significant portion of the database buffer cache has been written out to the system paging file" on one machine. Then maybe one or two days later Resource-Exhaustion-Detector/2004 events are following ending up with one or more processes crashing (e.g. MSExchange ActiveSync/1106 "Exchange ActiveSync has shut down because an unexpected critical error occurred...Exception type: System.OutOfMemoryException" or - worst case - Exchange log file replication crashing and DAG failover). If this happens one DAG member, the other Exchange servers follow at the same day with the same symptoms.
We did a number of corrective actions already: increased the VMs memory, moved all page files from an iSCSI volume to the the system drive (VMDK), increased the page file size (physical memory size plus 2 GB).
While observing the Perfmon counters today (while three of our systems showed the mentioned symptoms) I observed that "Available MBytes" frequently fall below the 100MB limit up to the time where Activesync crashed. That seemed to free memory again (a few GB in fact). The thing I do not understand so far is: alle the systems show Paging File "% Usage" counter values below 50% while the Resource-Exhaustion-Detector reports the low virtual memory condition. Can anyone explain that?
The processes consuming most of the memory are always store.exe, w3wp.exe and Microsoft.Exchange.RpcClientAccess.Service.exe.
Thanks for any help in advance.
Regards, Joern