Problem Statement
When using IP based storage over a converged 10GB network, should Jumbo Frames be used?
Requirements
1. Fully Supported storage
2. Maximum vSphere environment availability
3. Maximize performance where possible
Assumptions
1. Converged 10GB Network which is highly available
2. Two (or more) 10GB connections per ESXi host
Constraints
1. No dedicated network for IP storage traffic
Motivation
1. Simplify the environment
Architectural Decision
Do not use Jumbo Frames
Justification
1. Reduce the complexity in the environment for initial implementation
2. Simplify ongoing support / troubleshooting
3. For a Jumbo Frame to be transmitted without fragmentation, All devices end to end must support and be configured for Jumbo Frames
4. While there can be performance benefits of Jumbo Frames for IP Storage this is not generally seen across the board and depends on I/O types
5. Ensure IP storage packets are not fragmented or dropped by mis-configured devices or devices that do not support Jumbo Frames
6. Storage performance for the virtual environment will generally be constrained by the storage array controllers not the storage area network
7. Ensure packet fragmentation does not occur as all devices support a default MTU of 1500
8. Increasing the MTU will decrease the number of packets required for the same bandwidth but where the bottleneck is throughput (bytes) there will be minimal/no benefit
9. Jumbo Frames will only assist where the network is constrained at an interrupt level
Implications
1. IP Storage may have reduced performance in some circumstances compared to what Jumbo Frames may offer
Alternatives
1. Use Jumbo Frames
Relates Articles
1. Example Architectural Decision – Jumbo Frames for IP Storage (Use Jumbo Frames)
Thanks to Rob McNab (IBM) and Peter McCrystal (IBM) for their input into this example architectural decision.