As cluster sizes increase, it is important to understand the chance of multiple concurrent failure also increases and to architect solutions to ensure resiliency is maintained.
Because scalability is one of many strengths of the Nutanix Distributed Storage Fabric, Nutanix supported multiple data protection levels (RF2 and RF3) to ensure resiliency could be scaled with cluster size.
However using RF3 results in reducing the usable capacity to approximately 33% of the formatted capacity of the drives within the cluster which means it is sometimes considered undesirable.
But because some customers require the ability to support multiple concurrent node failures without the chance of data loss or unavailability, RF3 has been required.
Enter Nutanix Erasure Coding (EC-X)!
Now lets say you have a 32 node cluster where each node has 10TB RAW.
With RF3 we would have approx 3.33TB usable per node for a total of 106.56TB in the cluster.
With EC-X enabled (assuming EC-X has been applied to all data) the usable capacity would DOUBLE to 6.66TB per node and 213.12TB for the cluster.
Here’s how it works.
For RF3, the Nutanix Distributed Storage Fabric writes and maintains three copies of each piece of data. The below shows three copies of data “A” and “B”.
The below is a simplified example of what the Nutanix Distributed Storage Fabric looks like once EC-X is applied to RF3 data.
As you can see, we now support twice the amount of data as RF3 while still having dual parity. As a result, using RF3 + EC-X gives customers using large clusters MORE usable capacity than RF2 (~50% of RAW) while providing dual parity (which enables the loss of two nodes without data loss/unavailability).
Not bad for a software only upgrade!
So what do I recommend customers who are running 32 node or larger clusters?
1. For customers running RF3 already, Consider enabling EC-X.
2. For customers running RF2, consider enabling RF3 and EC-X
What is the computational overhead of enabling EC-X relative to RF3?
Hi John, As EC-X stripes the data in the same was as with RF2 the difference is one additional piece of parity which is written/maintained. In a 4+1 (RF2) vs 4+2 (RF3) the simple math tells us it would be approx 20%. The larger the stripe the lower the percentage overhead. But 20% for N+2 (as opposed to N+1) is pretty good IMO. Now 20% more overhead doesn’t mean 20% less performance, just potentially 20% more CPU per IO and for N+2. Hope that makes sense.