eBGP MultiHop Problem

  • 1
  • Problem
  • Updated 2 days ago
i am trying to set up peering between 2 cumulus devices seperated by 3 routers.
my setup is as follows :
CumulusA -> router->router->router->CumulusB

I have all the routes in between to reach the loopbacks of Cumulus A and B .
I am sourcing my ebgp session from the loopbacks of each.

the eBGP session is in ACTIVE state on both sides .
Any ideas ?
the loopback of CumulusA is 10.10.1.1
the loopback of CumulusB is 10.30.1.1

here is my config for the 2 Cumulus Boxes :
CumulusA-
router bgp 65000
  bgp router-id 10.10.1.1
  coalesce-time 1000
  neighbor 10.30.1.1 remote-as 65001
  neighbor 10.30.1.1 ebgp-multihop 255
  neighbor 10.30.1.1 update-source loopback

  address-family ipv4 unicast
    network 10.10.0.0/16


CumulusB-
router bgp 65001
  bgp router-id 10.30.1.1
  coalesce-time 1000
  neighbor 10.10.1.1 remote-as 65000
  neighbor 10.10.1.1 ebgp-multihop 255
  neighbor 10.10.1.1 update-source loopback

  address-family ipv4 unicast
    network 10.30.0.0/16
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb

Posted 5 days ago

  • 1
Photo of Jason Guy

Jason Guy, Employee

  • 1,712 Points 1k badge 2x thumb
There is likely no interface named "loopback" in linux, it will likely work if you set the update-source to "lo" or specify the address.
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb
Jason , thanks for the reply !
i updated my source to "lo" on both sides , still no go.
Also changed the source to the actual ip of the loopback on both sides, still no luck.
Below are my full configs. And again sourcing each loopback in a ping i can ping the other loopback , communication is there. I am running this in GNS3, btw.

CumulusA Config :
interface lo
  address 10.10.1.1/32

interface eth0

interface swp1
  bridge-access 100

interface bridge
  bridge-ports swp1
  bridge-vids 100
  bridge-vlan-aware yes

interface vlan100
  address 172.31.1.1/28
  vlan-id 100
  vlan-raw-device bridge

hostname DC1Cumulus

frr version 3.2+cl3u4

frr defaults datacenter

username cumulus nopassword

service integrated-vtysh-config

log syslog informational

vrf Default-IP-Routing-Table

router bgp 65000
  bgp router-id 10.10.1.1
  coalesce-time 1000
  neighbor 10.30.1.1 remote-as 65001
  neighbor 10.30.1.1 ebgp-multihop 255
  neighbor 10.30.1.1 update-source lo

  address-family ipv4 unicast
    network 10.10.0.0/16

ip route 0.0.0.0/0 172.31.1.2
line vty

dot1x
  mab-activation-delay 30
  eap-reauth-period 0

  radius
    accounting-port 1813
    authentication-port 1812

time

  zone
    Etc/UTC

  ntp

    servers
      0.cumulusnetworks.pool.ntp.org iburst
      1.cumulusnetworks.pool.ntp.org iburst
      2.cumulusnetworks.pool.ntp.org iburst
      3.cumulusnetworks.pool.ntp.org iburst

    source
      eth0

dns

  nameserver
    10.20.249.1

snmp-server
  listening-address localhost

#####################################
CumulusB Config:
interface lo
  address 10.30.1.1/32

interface eth0
  address dhcp

interface swp1
  bridge-access 100

interface bridge
  bridge-ports swp1
  bridge-vids 100
  bridge-vlan-aware yes

interface vlan100
  address 172.31.3.1/28
  vlan-id 100
  vlan-raw-device bridge

hostname cumulus

frr version 3.2+cl3u4

frr defaults datacenter

username cumulus nopassword

service integrated-vtysh-config

log syslog informational

vrf Default-IP-Routing-Table

router bgp 65001
  bgp router-id 10.30.1.1
  coalesce-time 1000
  neighbor 10.10.1.1 remote-as 65000
  neighbor 10.10.1.1 ebgp-multihop 255
  neighbor 10.10.1.1 update-source lo

  address-family ipv4 unicast
    network 10.30.0.0/16

ip route 0.0.0.0/0 172.31.3.2
line vty

dot1x
  mab-activation-delay 30
  eap-reauth-period 0

  radius
    accounting-port 1813
    authentication-port 1812

time

  zone
    Etc/UTC

  ntp

    servers
      0.cumulusnetworks.pool.ntp.org iburst
      1.cumulusnetworks.pool.ntp.org iburst
      2.cumulusnetworks.pool.ntp.org iburst
      3.cumulusnetworks.pool.ntp.org iburst

    source
      eth0

dns

  nameserver
    10.20.249.1

snmp-server
  listening-address localhost
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb
just as a side note , i also tried ebgp mh peering between the swp1 addresses on both CumulusA and B . thought maybe it was a bug with a loopback sourcing . but that didnt work either . and both swp1 vlan 100 addresses are able to reach each other. 
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb
Any other ideas ? It seems like a bug.
Photo of Jason Guy

Jason Guy, Employee

  • 1,712 Points 1k badge 2x thumb
Hi Scott,
Did you try activating the neighbor under ipv4 address-family?

router bgp 65000
  address-family ipv4 unicast
    neighbor 10.30.1.1 activate
router bgp 65001
  address-family ipv4 unicast
    neighbor 10.10.1.1 activate
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb
Jason, Thanks for the suggestion.
I did add it under both switches :

cumulus  2018-05-20 23:18:44.180932  net add bgp ipv4 unicast neigh 10.30.1.1 activate
cumulus  2018-05-20 23:18:46.566245  net commit


cumulus  2018-05-20 23:21:05.725257  net add bgp ipv4 unicast neigh 10.10.1.1 activate
cumulus  2018-05-20 23:21:08.623310  net commit

However, when i do a net show config bgp , the "activate" doesnt show up on either switches :

router bgp 65000
  bgp router-id 10.10.1.1
  coalesce-time 1000
  neighbor 10.30.1.1 remote-as 65001
  neighbor 10.30.1.1 ebgp-multihop 255
  neighbor 10.30.1.1 update-source lo
  neighbor 172.31.3.1 remote-as 65001
  neighbor 172.31.3.1 ebgp-multihop 255

  address-family ipv4 unicast
    network 10.10.0.0/16

Not sure what else to do ...

As a test i set up a simple normal eBGP peering session from Cumulus A to the upstream router, and that established with no problem. Just wanted to try a normal eBGP config to make sure i wasnt missing something.

I looked in the manual for Cumulus and configured just as it said.
i did the neighbor with both remote-as external and remote-as <AS Number>  and the only thing i added was 255 at the end of the ebgp multihop (ebgp-multihop 255).

Any additional suggestions would be greatly appreciated .

Thanks
-Scott
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb
i also added neighbor <neighbor address> disable-connected-check to both switches, still no go .
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb
also added - no bgp default ipv4-unicast

after adding that and adding the activate, the neighbor activate command shows up in the config

But still no dice :
 show ip bgp sum

IPv4 Unicast Summary:
BGP router identifier 10.10.1.1, local AS number 65000 vrf-id 0
BGP table version 0
RIB entries 1, using 152 bytes of memory
Peers 1, using 20 KiB of memory

Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd
10.30.1.1       4      65001       0       0        0    0    0    never       Active
Photo of Jason Guy

Jason Guy, Employee

  • 1,712 Points 1k badge 2x thumb
I noticed the static route is a default. I vaguely recall some caveat that BGP must have a route to the bgp speaker. Try configuring a more specific static route on each BGP speaker for the remote update source IP.
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb
Jason, that did it !is that a Cumulus specific thing ? I don't recall having to do that with Cisco anyways.
Also , on a side note i didnt see that mentioned in the Cumulus eBGP Multihop config guide.

Thank You Very Much !!!!-Scott
Photo of Jason Guy

Jason Guy, Employee

  • 1,712 Points 1k badge 2x thumb
Hi Scott,
No, this was true for IOS back when I studied for my CCIE. Doing a quick google search, I found this interesting article. Apparently this rule is implied in RFC 4271. The FRR developers confirmed that we certainly enforce this, but it can be turned off. If you think about it in a "real world" topology, would you really want to peer BGP using a default (the least specific route)? Glad it is working!
Photo of scott pledger

scott pledger

  • 170 Points 100 badge 2x thumb
Thanks Jason. Now that you mention it, I do remember having a specific route, vaguely. Been several years. Makes sense. Thank you so much.