In Part 1 and Part 2, we discussed how to size and configure Exchange VMs to meet CPU requirements. In Part 3 we will focus on virtual memory (vRAM).
As Exchange 2013 is quite RAM intensive and is not unusual to have memory requirements of >128GB RAM in larger deployments. As such, one of the first things we should consider is Virtual Machine maximums.
Luckily in recent years the maximum VM size in vSphere has increased and is no longer a constraint for virtualizing even the largest of Exchange environments.
The current maximum vRAM configuration per VM is shown below:
vSphere Virtual Machine RAM Maximums
Maximum vRAM per VM: 1TB (vSphere 5.0 or later)
Maximum vRAM per VM: 255GB (vSphere 4.1)
The key point here is memory is in no way a constraining factor when virtualizing Exchange even in older vSphere 4.1 deployments.
Memory Sizing
For maximum Memory performance, sizing the Exchange VM within a NUMA node gives the maximum benefit from NUMA locality, meaning the latency between the CPU and RAM is minimized.
In the event the memory requirements exceed the NUMA node, consider scaling out until you have at least 4 Exchange VMs (across 4 ESXi hosts) before scaling Exchange VMs up. This ensures higher resiliency and aligns with a Virtualization friendly scale out approach. Once the environment has 4 or more Exchange VMs, scaling up beyond the size of a NUMA node can be a good option to reduce the number of Exchange instances to manage and license without significantly impacting resiliency.
Memory Overcommitment
ESXi has excellent memory overcommitment capabilities which can work very well depending on the Operating system and application running within the guest. However Exchange is generally considered a Business Critical Application and as such, overcommitting memory for Exchange is generally not a good idea and should be avoided where possible.
Memory Reservations
For Exchange VMs, I recommended configuring the VM with “All Memory Locked” or in other words, a 100% memory reservation.
This has two advantages, the first being consistent memory performance for MS Exchange which is critical to ensure a great end user experience.
The second is the potentially large storage saving as the vSwap file is eliminated. For example, if an Exchange VM has 128Gb RAM and no memory reservation, a 128Gb vSwap file will be created by default in the same Datastore as the VMs .vmx file which could impact storage sizing and performance.
ESXi Host / Cluster Sizing Considerations
Exchange VMs are typically larger than the average VM, as a result they can consume a significant percentage of an ESXi hosts memory resources. As a result it is important to size your ESXi hosts to have sufficient RAM for the Exchange VMs.
As such in cases where the Exchange VM is sized to exceed the NUMA node, I recommend sizing ESXi hosts to have at least 25% more physical RAM than the vRAM assigned to your Exchange VMs.
Example: If your Exchange VM is assigned 96Gb, the ESXi hosts in the cluster should have at least 128Gb. This ensures memory for the hypervisor and other smaller VMs such as Domain Controllers to service things like the global catalog requirements for Exchange without contention.
Recommendations:
1. Set “All Memory Locked” (100% Memory Reservation) for Exchange VMs.
2. Where possible, size the Exchange VMs RAM within a NUMA node.
3. Where Exchange RAM requirements exceed that of the NUMA node ensure the size ESXi hosts to have at least 25% more RAM than the Exchange VM (or the largest vRAM VM in the cluster)
4. Ensure VMs vRAM is right sized after deployment to minimize waste (especially considering the recommendation to use memory reservations)
Back to the Index of How to successfully Virtualize MS Exchange.