If you’re setting up new server racks, then using a “top of rack” approach where you connect every rack to its own individual switch at the top of the rack and then link those into your broader network fabric produces a lot less cable clutter than a “structured cabling” approach tied to a centralised management area within each row or for the overall data centre. However, it could prove a lot more expensive over time.
Lifehacker’s coverage of Data Center World 2014 is presented by the Microsoft Cloud, providing flexible enterprise cloud solutions for business.
Todd Harpel, chair of the communication committee for the Communications Cable & Connectivity (CCCA) association, notes that choosing the right approach is essential to avoid ongoing maintenance hassles. “You need to prepare your data centre so you can connect without having to run new cables and make new connections all the time.”
In a presentation at Data Center World 2014 last week, Harpel presented numbers that suggest the initial savings on a top-of-rack approach, and potential advantages in environments with high levels of virtualisation, are ultimately swamped by higher maintenance costs. These were the numbers he presented:
The top-of-rack approach can make changes fiddlier. “When you have top-of-rack, it eliminates that central patching area, which makes it easy to make any connection from any switch to any server,” Harpel noted. Top-of-rack means you have to go to wherever that server is.”
“You’re deploying a lot more pieces of electronic equipment with top-of-rack. There are maintenance problems associated with that.” Heat rising is a particular concern: “The failure rate in the upper-third of a cabinet is three-times as much as lower down.”
The approach also makes upgrades more complex. If you update the switch in a centralised environment, every server will see performance benefits. With top-of-rack, each individual server switch requires upgrades. Some switches enforce the use of proprietary cable assemblies, which can also increase costs.
That doesn’t mean you should abandon top-of-rack approaches in all cases. “There’s no single solution that’s right for everyone,” Harpel said. “Your architecture changes with your needs.” In very small server rooms, or environments where servers have to be physically separated for security reasons, the benefits may outweigh the issues.
Rack picture from Shutterstock
Comments
3 responses to “Top Of Rack Cabling Can Be Costlier In The Long Run”
The picture in this article illustrates something I’ve always wondered. Why, when taking this approach, is the patch panel oriented so that the ports are on the opposite side to where the ports on the servers in the same rack are?
We tried putting a patch in the opposite way at work, and it turned out to be much easier to patch the servers into it. Means you only need to work on one side of the rack for one job, instead of both sides.
because all the pretty lights are at the front
Let’s not forget here that OH&S might cane you for a top of rack approach. In general you do not want to have to work for extended periods above your shoulder height. Any work over shoulder height means that you should be using a small set of steps (short ladder) or other appropriate means of raising the body safely above the floor.
I am not sure about this guy, but in Australia how many data centres have these available to 3rd party staff? None that I have worked in so far. I say great idea to plan your racks but they need to factor in that (if like me you are a long term member of the fraternity) I don’t like working on servers and switches that I need to be below kneeling height to work on. And as mentioned above working on, or lifting out a server in a rack that is too high on your own can spell disaster.
doesn’t most of this price difference just come down to server density/switch utilisation? If you’re buying $15k switches and only using half the ports on each (as seems to be the case in the example), you will spend more.
If you are flipping the orientation of the switch you need to be aware of maintaining your hot/cold aisle containment. You don’t want a switch facing the wrong way pushing heat into your cold aisle.