Problem Statement
In a vSphere environment, What is the most suitable disk provisioning type to use for the LUN and the virtual machines to ensure minimum storage overhead and optimal performance?
Requirements
1. Ensure optimal storage capacity utilization
2. Ensure storage performance is both consistent & maximized
Assumptions
1. vSphere 5.0 or later
2. VAAI is supported and enabled
3. The time frame to order new hardware (eg: New Disk Shelves) is <= 4 weeks
4. The storage solution has tools for fast/easy capacity management
Constraints
1. Block Based Storage
Motivation
1. Increase flexibility
2. Ensure physical disk space is not unnecessarily wasted
Architectural Decision
“Thin Provision” the LUN at the Storage layer and “Thin Provision” the virtual machines at the VMware layer
(Optional) Do not present more LUNs (capacity) than you have underlying physical storage (Only over-commitment happens at the vSphere layer)
Justification
1. Capacity management can be easily managed by using storage vendor tools such eg: Netapp VSC / EMC VSI / Nutanix Command Center
2. Thin Provisioning minimizes the impact of situations where customers demand a lot of disk space up front when they only end up using a small portion of the available disk space
3. Increases flexibility as all unused capacity of all datastores and the underlying physical storage remains available
4. Creating VMs with “Thick Provisioned – Eager Zeroed” disks would unnessasarilly increase the provisioning time for new VMs
5. Creating VMs as “Thick Provisioned” (Eager or Lazy Zeroed) does not provide any significant benefit (ie: Performance) but adds a serious capacity penalty
6. Using Thin Provisioned LUNs increases the flexibility at the storage layer
7. VAAI automatically raises an alarm in vSphere if a Thin Provisioned datastore usage is at >= 75% of its capacity
8. The impact of SCSI reservations causing performance issues (increased latency) when thin provisioned virtual machines (VMDKs) grow is no longer an issue as the VAAI Atomic Test & Set (ATS) primitive alleviates the issue of SCSI reservations.
9. Thin provisioned VMs reduce the overhead for Storage vMotion , Cloning and Snapshot activities. Eg: For Storage vMotion it eliminates the requirement for Storage vMotion (or the array when offloaded by VAAI XCOPY Primitive) to relocate “White space”
10. Thin provisioning leaves maximum available free space on the physical spindles which should improve performance of the storage as a whole
11. Where there is a real or perceved issue with performance, any VM can be converted to Thick Provisioned using Storage vMotion not disruptivley.
12. Using Thin Provisioned LUNs with no actual over-commitment at the storage layer reduces any risk of out of space conditions while maintaining the flexibility and efficiency with significantly reduce risk and dependency on monitoring.
13. The VAAI UNMAP primitive provides automated space reclamation to reduce wasted space from files or VMs being deleted
Alternatives
1. Thin Provision the LUN and thick provision virtual machine disks (VMDKs)
2. Thick provision the LUN and thick provision virtual machine disks (VMDKs)
3. Thick provision the LUN and thin provision virtual machine disks (VMDKs)
Implications
1. If the storage at the vSphere and array level is not properly monitored, out of space conditions may occur which will lead to downtime of VMs requiring disk space although VMs not requiring additional disk space can continue to operate even where there is no available space on the datastore
2. The storage may need to be monitored in multiple locations increasing BAU effort
3. It is possible for the vSphere layer to report sufficient free space when the underlying physical capacity is close to or entirely used
4. When migrating VMs from one thin provisioned datastore to another (ie: Storage vMotion), the storage vMotion will utilize additional space on the destination datastore (and underlying storage) while leaving the source thin provisioned datastore inflated even after successful completion of the storage vMotion.
5.While the VAAI UNMAP primitive provides automated space reclamation this is a post-process, as such you still need to maintain sufficient available capacity for VMs to grow prior to UNMAP reclaiming the dead space
Related Articles
1. Datastore (LUN) and Virtual Disk Provisioning (Thin on Thick)
a) Thin on thin provisioning does not provide significant storage savings compared to thin on thick – does it?
b) What did you mean by “Using Thin Provisioned LUNs increases the flexibility at the storage layer”?
Thin on Thin allows additional LUNs to be made available, not just for vSphere but other environments using the shared storage. As such, this increases the available “usable” storage to the environment and the flexibility for the stroage admins, for example when existing datastores have low free space, a new thin provisioned LUN can be presented without using real storage up front as is the case with Thick provisioned LUNs. If all LUNs are thick you have zeroed out all the blocks for those LUNs on the physical storage, and even when this space is left unused, nothing else can use it. Therefore Thick LUNs may lead to a higher level of wasted space compared to thin. Thanks for the comment and I hope I have clarified the example AD.