Reliability and Resiliency
Fibre Channel was purpose-built for connecting servers to storage devices. Fibre Channel SANs provide a high-performance expressway for storage traffic. For example, Fibre Channel incorporates a buffer credit mechanism to ensure reliable and in-order delivery of the data, host and target I/O devices that minimize CPU utilization with full offload capability, and switches with advanced analytics and inbuilt nameservers. Zoning is used to segregate and isolate target and initiator communications. In addition, new capabilities have been developed that allow switches and adapters to communicate with each other regarding the overall SAN environment. This helps identify and eliminate congestion in the fabric and provide self-healing capability within the SAN.
When trying to use other general-purpose protocols like Ethernet for storage connectivity, system administrators must go to great lengths to optimize the Ethernet environment for storage traffic. This includes mapping and masking for each connection on servers and switches; deploying lossless Ethernet networks to enable in-order delivery of storage information; and using link aggregation and establishing vLANs to provide segregation of data. This adds many steps that are not required with Fibre Channel and adds significant complexity to the network design while limiting scalability to one or two hops. This is especially true when deploying RoCEv2 RDMA, which is required to reduce the network latency. Fibre Channel had direct memory access with full offloads from day one. Here we are 20 years later, and the industry has come to the realization that Ethernet also should have direct access to system memory for I/O processing.
Cost and Complexity
When it was first developed, Fibre Channel was viewed as costly and complex. With hexadecimal addresses called worldwide names, charting out zones in the early days was a bit complex. The fact that SANs were completely dedicated to storage traffic and required two distinct paths in the fabric for high availability made them appear to be expensive as well.
However, in 2011, the concept of target-driven zoning came into play and allowed Fibre Channel switches to automate the zoning, greatly reducing the complexity of the Fibre Channel SAN. In addition, storage administrators have realized that even with Ethernet, the only way to deliver predictable storage performance is with a dedicated storage network. And today the TCO between high-performance Ethernet NICs, switches, cabling, and optics are comparable to Fibre Channel switches, adapters, and optics.
Bandwidth Debate
Ethernet proponents will tout the fact that 100Gbps and 200Gbps Ethernet is readily available and that Fibre Channel tops out at 64GFC today. While this is true, very few storage arrays exist with connectivity above 25Gbps for Ethernet and 32GFC for Fibre Channel. The fact that a switch or NIC can run at high bandwidth is irrelevant if the storage array and server PCIe can’t. In addition, 128Gbps Fibre Channel is in full development across the industry.
The standards groups for both Ethernet and Fibre Channel work from the same starting points for bandwidth. The same technology used for 100Gbps and 200Gbps Ethernet is already available for Fibre Channel and enabling its evolution to higher bandwidth. 2021 has seen the first deployments of 64GFC Fibre Channel in the data center for host and switch connectivity, but likely won’t become mainstream until sometime in 2023. By the time data center SANs move to higher bandwidth, Fiber Channel will be there, and with all of its unique benefits.
Benefits of QLogic® Fibre Channel HBA Technology from Marvell
Manageability and Troubleshooting
There’s no question that IT departments have the expertise to manage Ethernet networks. Heck, that is what all the internet and cloud traffic run on, making Ethernet skills table stakes in today’s enterprise. Likewise, there are plenty of software tools and utilities from a multitude of vendors to monitor and troubleshoot Ethernet issues that arise. This can create a challenge in that management tools vary by vendor across operating systems, switches and network adapters.
Fibre Channel isn’t that much different, however. As mentioned earlier, the setup and zoning of the storage fabric is far simpler with Fibre Channel than Ethernet, and because there are only two primary Fibre Channel Switch vendors and two Host Bus Adapter (HBA) vendors, there are only a few management utilities that the IT team need to be trained on. Diagnostic tools and software for Fibre Channel SANs are very comprehensive and new capabilities like Fabric Performance Impact Notifications (FPINs) enable the switch and HBA devices to interact with each other automatically to address issues like congestion and link integrity.
创新
Born on SCSI, innovation has always been in the DNA for Fibre Channel. In addition to increasing bandwidth in new generations of HBAs and switches, new capabilities continue to be added to the standards. Support for Hardware Root of Trust was added to prevent the insertion of malware in HBA firmware. A new standard for transmitting NVM Express™ over Fibre Channel (FC-NVMe) was introduced and all 16GFC enhanced HBAs and switches support this new protocol. FC-NVMe v2 was recently released to improve error correction handling and is now supported in select HBAs as well. Standards were further enhanced to support encryption in the Fibre Channel fabric and is available in select HBA offerings. All these capabilities further improve the performance and security of Fiber Channel technology.
Summary
There is no question that Ethernet has a place in data center storage. RDMA-enabled Ethernet networks are ideal for AI/ML solutions that require large data stores for a small number of servers. We see this today in JBOF and EBOF solutions available in the market. Ethernet is also ideal for software-defined storage or hyperconverged infrastructure (HCI) solutions, where multiple servers share captive storage resources. However, as new shared storage arrays get deployed, architects will find that Fibre Channel has all the necessary features and capabilities to remain the gold standard for SAN connectivity to these high-performance arrays. With 32GFC Fibre Channel readily available today and 64GFC Fibre Channel on the horizon, there is plenty of bandwidth available to run the business-critical workloads of today and tomorrow. With tens of millions of ports in production today, the proven reliability and scalability of Fibre Channel will continue to be the best option in the enterprise data center for connecting servers and storage together. At Marvell, we provide industry-leading QLogic Fibre Channel HBA technology to our customers. Our port-isolated architecture ensures predictable performance across each adapter port. Our unified driver makes deploying next-generation NVMe storage arrays easy, eliminating the need for additional drivers to be added to each server as competitive offerings require. Lastly, with full integration for management and troubleshooting with both Brocade® and Cisco® Fibre Channel switches and directors, QLogic adapters provide easy integration into any SAN fabric. For more information on QLogic Fibre Channel adapter technology from Marvell, visit www.marvell.com/qlogic.