Example Architectural Decision – Storage DRS Configuration for NFS Datastores

Problem Statement

In a vSphere environment, a NAS array is presenting Thin Provisioned NFS mounts (Datastores) to the vSphere environment. The storage has deduplication enabled across the datastores being used for the SDRS cluster. What is the most suitable configuration for SDRS to ensure the underlying storage efficiencies are not compromised while maintaining an even distribution of utilized capacity and I/O across all datastores?

Assumptions

1. vSphere 5.0 or later
2. NFS Based storage
3. NFS Mounts (Datastores) are Thin Provisioned
4. Deduplication is enabled on the array
5. VAAI is supported by the array and enabled across the vSphere environment
6. All datastores in a Datastore cluster are of the same RAID Type / Offer Similar performance due to having a similar spindle count
7. All datastores are presented to all hosts within the cluster

Motivation

1. Ensure storage efficiencies are not negatively impacted
2. Minimize the vSphere administrators workload where possible

Architectural Decision

Set the DRS automation setting to “No Automation (Manual Mode)”

  • Set “Utilized Space” threshold to 80%
  • Set “I/O latency” to 15ms
  • I/O Metric Inclution – Enabled

Advanced Options

  • No recommendations until utilization difference between source and destination is: 10%
  • Evaluate I/O load every 8 Hours
  • I/O Imbalance threshold  3

Justification

1. Setting Storage DRS to “No Automation (Manual Mode)” ensures that the administrator can confirm the recommendation will not Negatively impact the efficiency of Deduplication or  the thin provisioned NFS mounts
2. When creating a new Virtual Machine, in the “Ready to complete” window, Tick the “Show all storage recommendations” check box to review Storage DRS recommendations and override the recommendations where required
3. Where a VM is deduplicated on the source datastore, and it is moved to the destination datastore, this write activity is considered new data which will scanned by the post deduplication process which will use valuable CPU cycles on the array
4. “XCOPY” is not supported for NFS, as such, any Storage vMotion activity can only be offloaded to the array using the “Full File Clone” when a virtual machine is powered off.
5. Array level snapshots cannot be migrated with the VM using Storage DRS. If Virtual machines were automatically moved then the array level snapshot relasionship with the VM is broken and it cannot be leveraged
6. NFS datastores can be set to autogrow  by a predefined size in the event they reach a predefined utilization threashold
7. Where a significant I/O imbalance is detected by SDRS, the vSphere administrator can consider the impact of the Storage vMotion and where suitable apply the SDRS recommendation
8. SDRS still provides valuable “initial placement” for new virtual machines which will help avoid a situation where datastores are unevenly balanced from a capacity perspective
9. Storage DRS will still analysis I/O and where an imbalance is identified the vSphere administrator can choose to apply the SDRS recommendation to address the I/O imbalance

Implications

1. When selecting datastores for the datastore cluster, having VASA enabled allows the “System Capability” column to be populated in the “New Datastore Cluster” wizard to ensure suitable datastores of similar performance, RAID type and features are grouped together
2. A vSphere administrator will need to review SDRS recommendations

Alternatives

1. Use “Fully Automated”

Melbourne VMUG Presentation 26/7/2012 – Our VCDX Journey

Shane White and I gave a presentation to the Melbourne VMware User group this past Thursday evening.

For those who are interested, here is a copy of the presentation.

Josh Odgers & Shane White – MVMUG July 27th 2012 – VCDX Experience

The VCDX Application Process

I was asked by a person interested in attempting the VCDX if I could share my VCDX application / design, unfortunately as my application was based on an internal IBM project, it is strictly commercial in confidence.

However, I don’t think this is a huge problem as I can share my experience to assist potential candidates with their applications.

In the VCDX Certification Handbook and Application there are several sections, this post focuses on section 4.5 “Design Deliverable Documentation” and specifically the “A. Architectural design”.

Below is a screen shot of this section.

A piece of advise I shared in my post “The VCDX Journey” was that everything in your design is fair game for the VCDX panel to question you about. So for example if your design includes Site Recovery Manager OR vCloud Director , expect to answer questions about how your design caters for these products.

With that in mind, here are my tips.

Tip # 1 – Your design does not have to be perfect!

Don’t make the mistake of thinking you need to submit a design which follows every single “Best practice”, as this is very rare in reality. “Best Practice” is really a concept for VCP’s and to a lesser extent VCAP’s, a VCDX should be at a level of expertise too develop best practices, rather than follow.

Keep in mind, regardless of the architectural decision/s themselves,  You need to be able to justify them and align them to your “Requirements” , “Constraints” & “Assumptions” in both your documentation and the VCDX defense panel itself.

So you may have been forced to do something which is not best practice and that you wouldn’t recommend due to a “Constraint”. This is not a problem for the VCDX application, but be sure to fully understand the constraint and document in detail why the decision you made was the best opinion.

My design did not follow all best practices, nor was it the fastest or most highly available solution I could have designed. Ensure your aware of things which you could have done better, or could have changed if you did not have certain constraints, and document the alternatives.

I would suggest a design which complied with all best practices could be harder to defend, than one which had a lot of constraints preventing using best practices. As a candidate attempting to demonstrate your “Expert” level knowledge, working around constraints too meet your customer/s requirements would give you a better ability to show your thinking outside the square, so this goes for your documentation as well as the panel itself.

Tip # 2 – Don’t just fill out the VMware Solution Enablement Toolkit (SETs) template!

If your a VMware Partner, you will likely have access to VMware SETs. These a great resources which make doing designs easier (especially for people new to VMware architecture) however they are templates and anyone can fill out a template. As a VCDX applicant, you should be showing your “Expert” level knowledge / experience and innovation.

I personally have created my own template, which is a collaboration of numerous resources, including the SETs, but also has a lot of work I have created myself.

In my template I have a lot more detail than what can be found in the “SET” templates, and this I felt really assisted me in demonstrating my expert level knowledge.

For example I have a dedicated section for Architectural decisions where I had around 25 ADs for the design I submitted for VCDX, which covered not just specific VMware options, but Storage, Backup , network etc as these are all critical parts of a VMware solution. I could have have documented a lot more, but I ran out of time.

Tip # 3 – Document all your Requirements / Constraints / Assumptions and reference them.

Throughout your design, and especially your Architectural decisions, you should refer back to your Requirements, Constraints and Assumptions.

Doing this properly will assist the VCDX panel members who review your design to understand the solution. If the design document doesn’t give the reader a clear understanding of the solution then I would be surprised if you will be invited to defend.

During the VCDX defense, you should talk to how you designed too meet the Requirements and how the constraints impacted your design. You also should call out any assumptions, and discuss what risks or impacts these assumptions may have, this will be a huge help in your VCDX defense. so ensuring you have documented the ADs well for your application, is a big step towards your application being accepted.

Tip # 4 – Have your design peer reviewed

Where possible I always have my work reviewed by colleagues. Even VCAPs & VCDX’s make mistakes, so ensure you have your work reviewed. This is an excellent way to make sure your design makes sense, and is complete.

I touched on this in Tip # 3,  but make sure a person with zero knowledge of the solution, can read your design, and understand the solution. So get a review completed by somebody not involved with the project where possible.

Tip # 5 – Include information about Storage/Networking etc in your design

We all know, no VMware solution is complete without some form of Network & Storage, so ensure that your design has at least some high level details of the network & storage. This should assist you in other sections of your design document explaining your Architectural decisions, and give the reader a clearer picture of the whole environment.

Include diagrams of the end to end solution in an appendix so the reader can refer to them if any clarification is required.

Tip # 6 – Read the VCDX handbook and address each criteria.

As per the requirement document screen shot (above), the handbook actually tells you what VMware are looking for in your Architecture design.

It states “Including but not limited to: logical design, physical design, diagrams, requirements, constraints, assumptions and risks.”

In my design, Originally it didn’t in my opinion strictly meet all of the criteria, so I went back and added details to ensure I exceeded the criteria.

So in choosing what design to use for your application, my recommendation would be too not pick a small/simple design, but choose one which allows you to show your in depth knowledge and some innovation. This will make the application process a little more time consuming from a documentation point of view, but should increase your chance of success at the VCDX defense.

I hope this helps, and best of luck to anyone attempting the VCDX @ VMworld this year!