evpn type 5 routes

  • 1
  • Question
  • Updated 4 weeks ago
  • Acknowledged
  • (Edited)

I have setup bgp evpn peering between leaf1 and leaf4 in Cumulus VX , trying to get evpn type 5 routes exchanged for vrf RED ( followed the instructions as per "Announcing EVPN Type-5 Routes" from https://docs.cumulusnetworks.com/display/DOCS/Ethernet+Virtual+Private+Network+-+EVPN ). 

I couldn't get this to work,  not sure if i have configured incorrectly.


Any help is appreciated. 

is there an option to upload the configs over here ?
Photo of Vikram A

Vikram A

  • 208 Points 100 badge 2x thumb

Posted 2 months ago

  • 1
Photo of Vikram A

Vikram A

  • 208 Points 100 badge 2x thumb
I got it to work finally.  But i see icmp redirects when pinging from 
server1 [ 10.10.10.10 ] to server4 [ 10.10.100.10 ]

cumulus@server1:~$ ping 10.10.100.10
PING 10.10.100.10 (10.10.100.10) 56(84) bytes of data.
From 10.10.10.1: icmp_seq=1 Redirect Host(New nexthop: 10.0.0.14)
64 bytes from 10.10.100.10: icmp_seq=1 ttl=62 time=7.36 ms
From 10.10.10.1: icmp_seq=2 Redirect Host(New nexthop: 10.0.0.14)
64 bytes from 10.10.100.10: icmp_seq=2 ttl=62 time=5.94 ms
From 10.10.10.1: icmp_seq=3 Redirect Host(New nexthop: 10.0.0.14)
64 bytes from 10.10.100.10: icmp_seq=3 ttl=62 time=3.89 ms

cumulus@leaf1:~$ net show route vrf RED
show ip route vrf RED
======================
Codes: K - kernel route, C - connected, S - static, R - RIP,
       O - OSPF, I - IS-IS, B - BGP, P - PIM, E - EIGRP, N - NHRP,
       T - Table, v - VNC, V - VNC-Direct, A - Babel,
       > - selected route, * - FIB route

VRF RED:
K * 0.0.0.0/0 [255/8192] unreachable (ICMP unreachable), 00:56:20
C>* 10.10.10.0/24 is directly connected, vlan10, 00:56:20
B>* 10.10.100.0/24 [20/0] via 10.0.0.14, vlan10 onlink, 00:43:41

10.0.0.14 is the loopback on leaf4, is it normal to see icmp redirects ?
(Edited)
Photo of Eric Dong

Eric Dong

  • 762 Points 500 badge 2x thumb
seems like a confused kernel routing, would like to see what is cumulus's answer for this, in hardware based platform, cpu might not see this pkt at all, this could be cx specific issue?
Photo of Diane

Diane, Employee

  • 134 Points 100 badge 2x thumb
Hi Vikram,

This is Diane, TME with Cumulus.

We support EVPN Type 5 routes with the L3VNI with the symmetric IRB model today (CL 3.5).  According to your diagram and the output you provided, it looks like you are trying to run Type 5 routes over a L2VNI that should be already transporting the type 2 routes (which is not supported) - but I have not seen all your configs.  I would need to see those to recreate and determine what is causing the icmp redirects.

In this setup, you may be better off keeping the asymmetric model - and do your VXLAN Routing using Type 2 EVPN routes.   All that is required for this model is to configure the SVIs on the ToR.  You will need to configure both VNIs on both ToRs as the  asymmetric model routes only on the ingress ToR and thus traffic always travels on the destination VNI.

More information on symmetric vs asymmetric can be found here.

If you wish to use the symmetric model, we can do that also. It will also use Type 2 routes over the L3VNI for routing to a local host - you just won't need all the VNIs configured except the local ones on the local rack and the L3VNI and associated vlan. Type 5 routes are generally used for external routing only

Please let us know if you have any further questions.

Best Regards and have a great day!

Diane
(Edited)
Photo of Vikram A

Vikram A

  • 208 Points 100 badge 2x thumb
Thanks Diane for the explanation. I was kind of lost when it came to the L3VNI and might have misconfigured.

I was trying to simulate communication between 2 subnets which are in different DC’s but belong to the same tenant vrf using evpn. Will refer to the cldemo-evpn-symmetric you posted on github.
Photo of Diane

Diane, Employee

  • 134 Points 100 badge 2x thumb
Hi Vikram,

Thanks for the reply and explanation.

You can send the subnet as a Type 5 route with the "advertise-subnet" command - but this is mostly used to announce silent hosts when arp suppression is on.  After the /32 is learned (i.e. the destination host speaks) the Type 2 will be learned and the route will be a /32.  Since the routing table follows the longest match rule, the Type 2 route is used with the L3VNI.    CL3.5 does not yet support filtering the Type 2 routes with the L3VNI- this is coming in an upcoming release. 
Photo of Vikram A

Vikram A

  • 208 Points 100 badge 2x thumb
Hi Diane,

I re-did the configs in citc based on the configs from cldemo-evpn-symmetric (github) and i no longer see the icmp redirects.  It was a L3VNI misconfiguration on my part in the first attempt :(.



Thanks,
Vikram :)