Campus Switch Stacking

  • 2
  • Question
  • Updated 2 years ago
We're looking into using Cumulus Linux in our campus environment, and I was wondering what would be the best configuration solution for multiple switches daisy-chained to each other?
Photo of "B"

"B"

  • 1,110 Points 1k badge 2x thumb

Posted 2 years ago

  • 2
Photo of Jeff Haydel

Jeff Haydel, Alum

  • 122 Points 100 badge 2x thumb
Speaking from a pure, traditional, physical topology point of view I would think the ideal topology here would be to implement a tree instead of a chain in order to minimize the dependencies that will be affected by a random member dropping off (failing or rebooting.)


Now if you want to do something COOL though, consider an entire layer 3 to the edge closet, possibly using Cumulus' BGP/OSPF unnumbered, then a chain, or even a fully connect graph, makes more sense.  Why?  Well at layer 2 the loop mitigation protocol is spanning tree which simply disables/blocks all but a single path from each point of the network back to the defined network root (the switch in the network that wins the election and is likely at or near your core.)  But at layer 3, routing protocols would mitigate any loops or redundant paths and routing protocols can utilize those extra paths using ECMP (equal cost multi-path routing.)  This means that a routing protocol won't block or disable those duplicate paths but could potential utilize them for passing more traffic.




Chain:

Core <-> A - B - C - D - E

Tree:

Core <-> A
              /   \
           B      C
          /           \
        D             E

Fully Connected Graph:

Core <->   A  -  B
               /   \  \ / /  
             C - D -E
             |---------|

I hope this helps!

Jeff H
SE, southeast US
Photo of "B"

"B"

  • 1,110 Points 1k badge 2x thumb
Hey Jeff, thank for the reply!  I will keep that in mind, but maybe I should have phrased it a different way.  What about stacking the switches to look like one switch?  Right now we have a couple of Cisco 3850 that we stack to seem as one.  If we were to replace those switches, could we configure Cumulus Linux to give us the same result using white boxes?
Photo of Kevin Witherstine

Kevin Witherstine, Employee

  • 770 Points 500 badge 2x thumb
"B", no stacking as you'd know it with the 3850. Typically interconnect boxes with trunk ports and link aggregation.

If you're end goal is to simply create a single configuration point, centralized automation would give you a similar abstract for controlling large numbers of ports and devices without having to be concerned about stacking devices.
Photo of "B"

"B"

  • 1,110 Points 1k badge 2x thumb
Thanks Kevin!
Photo of Srivats Iyer

Srivats Iyer, Alum

  • 100 Points 100 badge 2x thumb
Photo of "B"

"B"

  • 1,110 Points 1k badge 2x thumb
Thanks Srivats!
Photo of Jeff Haydel

Jeff Haydel, Alum

  • 122 Points 100 badge 2x thumb
Thanks for replying B!  I see now that you are specifically asking about stacking more so than chaining?  Well the short answer to your question is no, Cumulus does not support stacking.  

The longer answer is to look at why and how Cumulus provides a better solution to the problems that the inventors were trying to solve with the release of stacking.  (Warning!  History and sales content to follow) Stacking solves 3 distinct problems:  1) ease of management through the use of a single IP address or management plane, 2) sharing of the control plane to allow a closet to survive a switch failure and 3) multi-chassis uplink redundancy.  

Let's tackle these 3 problems 1 by 1:

1) ease of management:  There is an undoubtably elegant beauty to interfacing with a single IP address to manage an entire edge closet's worth of switches.  But wouldn't it be even cooler to manage an entire closet, floor, building, or campus in the same concept.  Stacking keeps this concept of single mgmt IP to a physical location, but Cumulus, through the use of automation and orchestration allows for unlimited "stacking" by closets, floors, buildings, or even use cases like wireless switches, VOIP switches, or Dell switches.  Automation and orchestration take the concept of stacking and moves it into the 21st century by allowing for dynamic stacking at the admin's discretion.  I see stacking here like the janitor's large key ring hanging on the hip.  With the inventions from Nest, Kevo, and the like these key rings are a dying breed as we just keep our "keys" on our phone!

2) Switch failure survival of the control plane:  This is another solution for a problem of a by gone era.  "Back in the day" when switches were the latest and greatest, the CPU costs for a switch were expensive.  And so companies looking to save costs would buy one or two expensive "stack switch controllers" and then multiple less expensive "stack member switches."  Those member switches wouldn't have the beef to handle entire ARP or CAM tables but they could pass traffic to and from the switch controllers that could.  Nowadays the x86 and ARM CPU's are less expensive so this is no longer a concern.

3)  Uplink redundancy:  This problem is a direct result of that ancient technology spanning tree.  Spanning tree, as I referenced above, disables all paths except the single path that it computes is best for a path.  This means that two paths, redundant paths, from the same stack or chain of switches guarantees that there will be at least one link blocked down.  This isn't exactly a problem but it isn't efficient.  What is a problem in this scenario is the time it takes that blocked link to come up after failure, and even worse, the time it takes the original link to go back down when the original link is operational once more.  This solution also completely invalidates the concept of aggregating those 2 links together to achieve greater uplink output.   Cumulus' solution here is utilizing layer 3 to the edge and then achieving ECMP as discussed above.  

/History&Sales

So I hope you agree from the above solution that while Cumulus doesn't support stacking it supports a series of superior solutions to the same problems that stacking tried to solve way back when.

Thanks,

Jeff H
Photo of Jeff Haydel

Jeff Haydel, Alum

  • 122 Points 100 badge 2x thumb
Oops.  While I was writing my lengthy response it looks like Kevin and Srivats got something out faster!  Well better multiple correct answers than no answers at all!

Thanks,

Jeff H
Photo of "B"

"B"

  • 1,110 Points 1k badge 2x thumb
As you say, better to have more than none at all!  I really do appreciate the feedback and the quick responses by you and your team!!!