If you’re setting up new server racks, then using a “top of rack” approach where you connect every rack to its own individual switch at the top of the rack and then link those into your broader network fabric produces a lot less cable clutter than a “structured cabling” approach tied to a centralised management area within each row or for the overall data centre. However, it could prove a lot more expensive over time.
Todd Harpel, chair of the communication committee for the Communications Cable & Connectivity (CCCA) association, notes that choosing the right approach is essential to avoid ongoing maintenance hassles. “You need to prepare your data centre so you can connect without having to run new cables and make new connections all the time.”
In a presentation at Data Center World 2014 last week, Harpel presented numbers that suggest the initial savings on a top-of-rack approach, and potential advantages in environments with high levels of virtualisation, are ultimately swamped by higher maintenance costs. These were the numbers he presented:
The top-of-rack approach can make changes fiddlier. “When you have top-of-rack, it eliminates that central patching area, which makes it easy to make any connection from any switch to any server,” Harpel noted. Top-of-rack means you have to go to wherever that server is.”
“You’re deploying a lot more pieces of electronic equipment with top-of-rack. There are maintenance problems associated with that.” Heat rising is a particular concern: “The failure rate in the upper-third of a cabinet is three-times as much as lower down.”
The approach also makes upgrades more complex. If you update the switch in a centralised environment, every server will see performance benefits. With top-of-rack, each individual server switch requires upgrades. Some switches enforce the use of proprietary cable assemblies, which can also increase costs.
That doesn’t mean you should abandon top-of-rack approaches in all cases. “There’s no single solution that’s right for everyone,” Harpel said. “Your architecture changes with your needs.” In very small server rooms, or environments where servers have to be physically separated for security reasons, the benefits may outweigh the issues.
Rack picture from Shutterstock