Link Aggregation across two MLAG'd switch stacks

  • 1
  • Question
  • Updated 2 days ago
Hi,

My question is regarding how it would be best to link two pairs of switches together, with a minimum number of connections.

I've got links between the four of my switches as described in the image below:


Switches A and B are configured with two links into a MLAG, switches C and D are also setup in the same configuration.

I have two additional links, from switch A to C and B to D, but I'm not sure how best to configure these; can anybody suggest what you think my best course of action may be please?

Many Thanks,
Adam.
Photo of Adam Carrgilson

Adam Carrgilson

  • 104 Points 100 badge 2x thumb

Posted 1 week ago

  • 1
Photo of Nipul Shah

Nipul Shah, Employee

  • 380 Points 250 badge 2x thumb
Hi Adam,

Would suggest atleast one line from C to each A and B, and similarly from D to A and B for redundancy. 
Irrespective of the number of links, switch pair A & B for an MLAG pair, and switch pair C & D form another MLAG pair. 
The links between these pairs need to be in a bond. So, for your diagram, swp14 on A & B will be a bond with the same clag-id, and swp14 on C & D will be a bond with a shared clag-id.

For a configuration example, on your cumulus switch or VX, run the following:
sw:~$ net example clag l2-with-server-vlan-trunks

That will give a topology diagram, similar to your setup, and configurations as well.
Photo of Nipul Shah

Nipul Shah, Employee

  • 380 Points 250 badge 2x thumb
Adam, logically speaking it's a single bond between 2 clag-pairs
i.e. clag-A-B ===bond=== clag-C-D
So, one bond is configured on each switch. Pay attention to the clag-sys-mac and clag-id
clag pair switch A & B will share one clag-sys-mac, say 44:38:39:FF:00:00
and, clag pair switch C & D will share a different clag-sys-mac, say 44:38:39:FF:00:01

Taking the configuration from the "net example" command I provided previously, in your case, the bond configuration would be something like this:

#Between switch A and B, #the clag-id must be the same for the bond<br>swA# net add clag port bond bond-leaf-0A-0B interface swp14 clag-id 1<br>swB# net add clag port bond bond-leaf-0A-0B interface swp14 clag-id 1<br> #Between switch C and D, #the clag-id must be the same for the bond<br>#clag-id is only signification for a given clag pair.<br>#so clag-id for the bond on switch C and D can be different #from what was used for the bond on switch A and D<br>swC# net add clag port bond bond-leaf-0C-0D interface swp14 clag-id 1<br>swD# net add clag port bond bond-leaf-0C-0D interface swp14 clag-id 1
(Edited)
Photo of Nipul Shah

Nipul Shah, Employee

  • 380 Points 250 badge 2x thumb
Repasting the configuration, as edit failed.

#Between switch A and B, 
#the clag-id must be the same for the bond
swA# net add clag port bond bond-leaf-0A-0B interface swp14 clag-id 1
swB# net add clag port bond bond-leaf-0A-0B interface swp14 clag-id 1
#Between switch C and D, 
#the clag-id must be the same for the bond
#clag-id is only signification for a given clag pair
#so clag-id for the bond on switch C and D can be different 
#from what was used for the bond on switch A and D
swC# net add clag port bond bond-leaf-0C-0D interface swp14 clag-id 1
swD# net add clag port bond bond-leaf-0C-0D interface swp14 clag-id 1
Photo of Adam Carrgilson

Adam Carrgilson

  • 104 Points 100 badge 2x thumb
Hi Nipul,

So I've finally managed to get my interfaces between across campus live, and I think I've discovered what you alluded to regarding the clag-sys-mac.

My net show clag now shows a status of: lacp partner mac mismatch.

I've already configured multiple sys-mac details for the peerlink, do I need an additional sys-mac as you say for this new bond-leaf-link?

Many Thanks,
Adam.
Photo of Nipul Shah

Nipul Shah, Employee

  • 380 Points 250 badge 2x thumb
Adam,

the clag-sys-mac is a clag level configuration that is done only on the peerlink bond, and not on other bonds. 
so, eg: on both switch A & B, you would use:
#switchA
net add clag peer sys-mac 44:38:39:FF:00:00 interface swp15-16 primary backup-ip <switchB-eth0-ip>
#switchB
net add clag peer sys-mac 44:38:39:FF:00:00 interface swp15-16 secondary backup-ip <switchA-eth0-ip>

Then on switch C & D, note the different sys-mac:
#switchC
net add clag peer sys-mac 44:38:39:FF:00:01 interface swp15-16 primary backup-ip <switchD-eth0-ip>
#switchD
net add clag peer sys-mac 44:38:39:FF:00:01 interface swp15-16 secondary backup-ip <switchC-eth0-ip>
Photo of Adam Carrgilson

Adam Carrgilson

  • 104 Points 100 badge 2x thumb
Hi Nipul,

Thank you so much for your assistance, I think I now have a working configuration!

Many Thanks,
Adam.
Photo of Sergei Hanus

Sergei Hanus

  • 476 Points 250 badge 2x thumb
Maybe, not a relevant reply, but have you tried to re-architect this solution so it doesn't require MLAG? For example, to move completely to L3 topology and use routing protocols for providing redundant paths?

Imagine troubleshooting your MLAG setup one day and think, how hard it could be.
(Edited)