Should I use optical fiber or twinax cable?

Earlier this week I received a question regarding EMC’s recommendations for using twinax cable.  My response is included below, but I need to tip my hat to Mark Lippitt for reviewing, correcting, and adding a couple of different considerations that I overlooked.  One other point worth mentioning is that we are working on an update for the February edition of the EMC FCoE TechBook that will also include the following information.

Question: “Should I use optical or twinax connections when connecting my CNAs to the ToR switch?”

Answer: You can use optical connections for any FCoE link.  The downside is that they are more expensive and use more power.  The upside is that you don't have to worry about distance (up to 300 meters) and, perhaps more importantly, you don't have to worry about which vendor’s cable you use…

Typically, we see twinax (copper) being used between the server and the Top of Rack (ToR) switch.  The upside in this configuration is that the twinax connection is much cheaper than the cost of an optical link.  The downside is that you are limited in distance and there‘s also some cable interoperability issues you’ll need to deal with.

In regards to active versus passive twinax cables, it depends on what you are connecting together:

Note: The following information assumes that either FOS 6.4.1 (or later) or NX-OS 4.2(1)N2(1) (or later) is being used.

• If you are connecting a CNA to a Cisco ToR switch (Nexus) and the cable length is 1, 3, or 5m, you can use a Cisco-supplied 1, 3, or 5m passive twinax cable or a EMC active twinax 1, 3, or 5m cable.
• If you are connecting a CNA to a Cisco ToR switch (Nexus) and the cable length is 7 or 10m, you need to use a Cisco-supplied 7 or 10m active twinax cable.
• If you are connecting a CNA to a Brocade ToR switch (8000) and the cable length is 1, 3, or 5m, you can use a Brocade-supplied 1, 3, or 5m active twinax cable or an EMC-supplied 1, 3, or 5m active twinax cable.

The above assumes that you are using a CNA that supports a given flavor of twinax.  As of right now, the QLogic 815x and Emulex OCe1012, support either active or passive twinax from both Cisco and QLogic.  The Brocade CNAs support active twinax from Brocade or EMC’s active twinax cable. 

If you want to connect a Brocade CNA to a Cisco ToR switch (Nexus), you could use the EMC active twinax cable.  Otherwise, you'll need to use optical since the Cisco switch will log an error if you attempt to use the Brocade cable.

If you want to attach EMC native FCoE storage (CLARiiON) to a Brocade switch using twinax you can use either:

• EMC's 1, 3, or 5m active twinax cable; or
• Brocade’s 1, 3, or 5m active twinax.

If you want to attach EMC native FCoE storage (CLARiiON) to a Cisco switch using twinax:

• You can use EMC's 1, 3, or 5m active twinax cable.
• You CANNOT use Cisco’s passive 1, 3, or 5m passive twinax.
• We are testing Cisco’s 7, 10m active twinax with native FCoE storage, but this is not completed yet.

With regard to compatibility algorithms:

• Cisco will always bring up a cable if it can. If it does not recognize the cable, it will post an error message to the system log, but bring up the cable.
• Brocade and EMC must recognize the cable. If they do not recognize the cable, they will not bring up the link.