How to successfully Virtualize MS Exchange – Part 1 – CPU Sizing

Part 1 will focus on CPU sizing for Exchange Mailbox (MBX) or Multi Server Role (MSR) deployments.

The Exchange 2013 Server Role Requirements Calculator v6.6 should be used to size the VM to ensure sufficient performance for the specific Exchange deployment.

One key input for the calculator is the “SpecInt2006 Rate Value” which can be found on the “Input” tab of the calculator  (shown below).

SPECint2006Input

To find the SpecInt2006 Rate Value for your specific CPU, I recommend using the  Exchange Processor Query Tool which allows you to enter the Processor Model number of your servers and query the Spec.org database for the rating of your CPU.

Note: This tool is applicable to Exchange 2010 and 2013 deployments despite the tool being titled “Exchange 2010 Processor Query Tool”.

To do this, enter the model number of your CPU (example E5-2697 v2 shown below) and press query.

ProcQueryTool01

The calculator will then return the list of tested server in the right hand side of the spreadsheet an example of this is shown below.

ProcQueryTool02

The SpecInt2006 result for your CPU is highlighted in Orange in the “Result” column.

At this stage the drop box in Step 4 allows you to choose the number of physical cores planned to be used and it will then return the average result of all tested servers.

ProcQueryTool03

The above result for example assumes a Dual Socket physical server with 12 core Intel E5-2697 v2 processors.

As we are discussing Virtualizing Exchange, Step 7 is applicable.

Here the tool allows you to enter the overcommitment (vCPU to Physical Core) and the number of vCPUs (called virtual processors in the spreadsheet) which then results in what the spreadsheet calls

“Virtual mailbox server SPECint2006 Rate Value” shown below in Orange.

ProcQueryTool04

The calculator makes the assumption that CPU overcommitment of 2:1 degrades performance by 50% which is not strictly true, but can be used as general guidance that high levels of CPU overcommitment which may lead to CPU contention are not recommended for MS Exchange deployments. It is important to note, CPU overcommitment ≠ CPU contention, although the higher the overcommitment, the higher the possibility of contention (CPU Ready).

Now that we have the SPECint2006 Rate Value, this can be entered into the Primary and Secondary (if applicable) field of the Exchange 2013 Server Role Requirements Calculator (shown below).

SPECint2006ratePrimSec

The SPECInt2006 value is for the physical processor, which if it supports Hyper-threading (HT), means the rating includes the performance benefit of HT. The key point here is using just physical cores for sizing means your VM will not get the full performance of the SPECint2006 rating, it will be slightly less. This will be discussed in more detail in Part 2 – vCPU configurations.

The “Processor Cores / Server” field should be populated by the physical cores intended to be used.

While the “Processor Cores / Server” value does not impact the CPU utilization calculations, entering the “Processor Cores / Server” allows the calculator to report the number Processor Cores Utilized as shown below from the “Role Requirements” tab.

ServerConfigBefore2

The number of cores utilized helps calculate the number of vCPUs required for the Exchange VM. If the “Server CPU utilization” is much lower than 80% (recommended maximum), the “SPECInt2006 rate value” and “Processor Cores / Server” can be reduced.

Example: If the calculator reports Server CPU Utilization at 40% and the CPU Type is Intel E5-2697 v2 with 12 physical cores with a SPECint2006 rating of 479. The Virtual Machine should be sized with 6 vCPU. To confirm this the SPECint2006 rating for Primary and Secondary (if applicable) field of the Exchange 2013 Server Role Requirements Calculator can be reduced by 50% (from 479) too 239.5 which will result in the calculator reporting Server CPU Utilization at 80%.

Another option is to review the number of Mailbox Servers are configured, and where the utilization is low as in the previous example 40%, you could choose to “scale up” each of the Exchange VMs. To do this, change the highlighted field on the “Input” tab of the calculator to 4, and you will see the utilization under “Server configuration” increase (on the role requirements tab) to 80%.

MBXserversPerDAG

Scaling up reduces the number of Windows/Exchange instances licenses and ongoing maintenance (such as patching) required, but also increases the failure domain and impact of a failure so this decision needs to not only be a architectural/technical one, but a business decision.

As a general rule, I recommend customers scale out until they have 4 or more Exchange VMs (across 4 or more ESXi hosts), then scale up (and out) as required. This ensures the impact of a server failure is 25%, compared to 50% if it was a scaled up Exchange server deployment with only 2 VMs.

An important consideration for any business critical application deployment is the scalability of the solution. In this case, when discussing virtualizing one or more Exchange servers, the Virtual Machine maximums are critical.

The below shows the maximum vCPUs supported for a VMware based virtual machine.

vSphere Virtual Machine CPU Maximums

Maximum vCPUs: 64 (vSphere 5.1 or later)
Maximum vCPUs: 32 (vSphere 5.0)
Maximum vCPUs: 8 (vSphere 4.1)

The above numbers are dependant on the physical hardware chosen.

Recommendations for CPU sizing:

1. CPU overcommitment be less than 2:1, and ideally 1:1 for hosts servicing Exchange workloads. This will be discussed further in this series.

Use the vSphere Cluster Sizing Calculator to confirm overcommitment ratios for your cluster or to validate your design.

2. Size Exchange Server VMs to less than 80% CPU Utilization

This allows for burst activity such as increased load or DAG failovers.

3. Scale up a single Exchange VM per ESXi host as opposed to running multiple smaller Exchange VMs per host.

4. Do not oversize Exchange VMs Day 1, Size for Day 1 demand and scale vCPUs as required (which can be done quickly and easily thanks to the virtual layer).
5. CPU reservations do not solve CPU scheduling contention (a.k.a CPU Ready). CPU reservations should not be required in properly sized environments.
6. Size Exchange VMs using Physical Cores and assume no benefit from HT
7. Leave HT turned on at the ESXi layer

Back to the Index of How to successfully Virtualize MS Exchange.

Integrity of I/O for VMs on NFS Datastores – Part 4 – Torn Writes

This is the fourth part of a series of posts covering how the Integrity of Write I/O is ensured for Virtual Machines when writing to VMDK/s (Virtual SCSI Hard Drives) running on NFS datastores presented via VMware’s ESXi hypervisor as a “Datastore”.

This part will focus on Torn Write I/O.

As a reminder from the first post, this post is not talking about presenting NFS direct to Windows.

Some of you are probably wondering “What is a Torn Write”?

A Torn write can occur if there is a problem (e.g.: Power or HW failure) during a multi sector block being written.

The below shows what a Torn Write looks like, which is basically where part of data A and B remain after a Torn write, resulting in corrupted data.

Torn Write

Image Source: Silent data corruption in disk arrays: A solution

The article Toward I/O-Efficient Protection Against Silent Data Corruptions in RAID Arrays describes a Torn Write (I/O) as:

Torn write: When a disk write is issued to a chunk, only a
portion of sectors in the chunk are successfully updated,
and the chunk contains some stale sectors in the end part.

The issue with the write I/O being written across multiple sectors is that in the event a power outage impacting the write back cache or a hardware issue such as a drive failing, the I/O may be partially written (or “Torn”). This means the data was not fully written, but some data was written overwriting the existing data causing corruption.

In this case, if the storage solution provides a write acknowledgement and the data is partially or not written to persistent media this results in what is known as silent data corruption as data being read back will be part of the new data and part of the old data.

It should be noted RAID does not protect against Torn writes, nor can it help correct the situation once it has occurred.

The next question is, does the issue of Torn writes impact VMs on ESXi backed by NFS datastores. The answer is, Yes because Torn Writes can potentially occur on any storage solution regardless of the abstracted storage protocol.

So do Torn Writes occur VMs on ESXi backed by NFS datastores? The answer again would be Yes, but importantly, this would not be as a result of anything at the hypervisor layer, it would be as a result of a failure impacting the underlying storage.

Note: This issue equally impacts block and file based storage presented to ESXi, so it is not a NFS specific issue.

So what is required to provide protection against Torn Writes?

The best method to protect against Torn Writes is to use checksums, specifically Block level checksums which can check the integrity of writes which span multiple sectors, therefore in the event of a torn write, the checksum will fail and a write acknowledgement will not be sent. The important fact here is the underlying storage is responsible for this process, not ESXi , the VMDK or storage protocol (FC,FCoE,iSCSI, NFS!) presenting the storage to ESXi.

In summary, Torn Writes are not an issue with VMs running on ESXi backed by NFS datastores where the underlying storage performs Block level checksums.

I have requested VMware create a Knowledge base article on Torn Writes for formal reference and will update this post with the reference if/when this is done.

In part five, I will discuss Data Corruption.

Integrity of Write I/O for VMs on NFS Datastores Series

Part 1 – Emulation of the SCSI Protocol
Part 2 – Forced Unit Access (FUA) & Write Through
Part 3 – Write Ordering
Part 4 – Torn Writes
Part 5 – Data Corruption

Nutanix Specific Articles

Part 6 – Emulation of the SCSI Protocol (Coming soon)
Part 7 – Forced Unit Access (FUA) & Write Through (Coming soon)
Part 8 – Write Ordering (Coming soon)
Part 9 – Torn I/O Protection (Coming soon)
Part 10 – Data Corruption (Coming soon)

Related Articles

1. What does Exchange running in a VMDK on NFS datastore look like to the Guest OS?
2. Support for Exchange Databases running within VMDKs on NFS datastores (TechNet)
3. Microsoft Exchange Improvements Suggestions Forum – Exchange on NFS/SMB
4. Virtualizing Exchange on vSphere with NFS backed storage

The VCDX candidates advantage over the panellists.

As the candidate, you submit a VCDX application based on a project you have worked on from start to finish and in most case, lead from a technical/architectural perspective.

You therefore should have:

  • Had Initial discussions with the customer about requirements.
  • Lead or been involved in Design workshops
  • Considered design decisions
  • Documented the detailed design along with implementation & test plans etc
  • Either overseen or been actively involved with implementation

As a result of the above, you have spent many hours, potentially hundreds or thousands of hours depending on the size of the project and you will have intimate knowledge of the design & solution.

On the other hand, VCDX panellists, while they are experts in the field, have been given your application, including the design and a very limited amount of time to review and prepare for the defence.

As a result, the VCDX candidate has a HUGE advantage over the panellists!

So in a defence, who is the expert in the room on the design? The Candidate!

As a result, the candidate should be an expert in the design being presented and answering questions from the panel about the design should not be intimidating.

To all VCDX candidates, understand that you have a major advantage and go defend your design with confidence!