cumulus box didn't respond to traceroute, what did I miss?

  • 1
  • Question
  • Updated 2 years ago
cumulus box didn't respond to traceroute, did tcpdump br0 interface, I can see ttl expire pkt hit cpu, but seems kernel didn't bother to send icmp pkt back to source,

what did I miss? is this cumulus linux default behaviour?
Photo of Eric Dong

Eric Dong

  • 772 Points 500 badge 2x thumb

Posted 2 years ago

  • 1
Photo of Sean Cavanaugh

Sean Cavanaugh, Alum

  • 3,380 Points 3k badge 2x thumb
Is there an IP address on br0 (cumulus@switch$ip addr show br0) and is that IP the gateway for traffic on that bridge. It won't show as a hop unless it's a L3 hop.
Photo of Eric Dong

Eric Dong

  • 772 Points 500 badge 2x thumb
yes, br0 is host's gw ip, 
Photo of Sean Cavanaugh

Sean Cavanaugh, Alum

  • 3,380 Points 3k badge 2x thumb
VX or Cumulus Linux? MLAG?
Photo of David Marshall

David Marshall, Employee

  • 530 Points 500 badge 2x thumb
Can you post output from "ifquery -a"?
Photo of Eric Dong

Eric Dong

  • 772 Points 500 badge 2x thumb
it is Cumulus Linux.
just to be clear, only cl didn't reply, tracert was able to finish. ( but cl show up as * * * )
~~~
...
auto br-tag1iface br0
address 172.16.0.5/30
bridge-stp on
bridge-ports swp1 swp2

auto swp1
iface swp1

auto swp2
iface swp2


on host side.
~~~
test@host:~/bin$ ip route show
default via 172.16.0.5 dev eth0
...
Photo of Eric Pulvino

Eric Pulvino, Official Rep

  • 4,082 Points 4k badge 2x thumb
There is no sysctl variable to explicitly disable the sending of IP unreachables, so if the kernel is seeing the expired TTL packet it should appropriately generate a response. I wonder if some of the control plane policing might be playing a part here. A few thoughts here:
-Try removing all Control Plane Policing with "sudo cl-acltool -F all" to see if responses are being filtered for some reason.
-Could you show the output of "ip addr show"
-What does the tcpdump show on the switchport which faces the host? Is any kind of response seen what so ever?
-What kind of traceroute traffic is the host sending (some use UDP, some use ICMP, some can be set to use TCP)?
-Have you tried manipulating the kind of traceroute traffic that is sent? (i.e. if you're using UDP, can you try ICMP) with the same results etc?
-What release are you running? (" cat /etc/lsb-release")

I will start poking around with this in the lab when I return from the holidays to see if I can recreate what you're seeing.
Photo of Eric Dong

Eric Dong

  • 772 Points 500 badge 2x thumb
I find it, I don't have a routed kernel stack, ( ip_forward is 0 )
sorry for the confusing, it is not cumulus issue.
I need ip_forward=1 to make cumulus linux show up in traceroute.


while I am debugging, I noticed cumulus is setting these traffic to class 7.
what are these? thanks 


???
~~~
SETCLASS   tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:5342 SETCLASS  class:7
POLICE     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp dpt:5342 POLICE  mode:pkt rate:2000 burst:2000
SETCLASS   tcp  --  0.0.0.0/0            0.0.0.0/0            tcp spt:5342 SETCLASS  class:7
POLICE     tcp  --  0.0.0.0/0            0.0.0.0/0            tcp spt:5342 POLICE  mode:pkt rate:2000 burst:2000



????
~~~~
SETCLASS   udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:10001 SETCLASS  class:7
POLICE     udp  --  0.0.0.0/0            0.0.0.0/0            udp dpt:10001 POLICE  mode:pkt rate:1000 burst:1000
Photo of Eric Pulvino

Eric Pulvino, Official Rep

  • 4,082 Points 4k badge 2x thumb
The classes "set the system internal class of service queue configuration to value" see our ACL documentation here --> https://docs.cumulusnetworks.com/disp...
Photo of Eric Dong

Eric Dong

  • 772 Points 500 badge 2x thumb
appreciate the link but it didn't talk about what kind of traffic is udp 10001 and what is tcp 5342.

I guess I didn't make my question clear, I am just curious why these traffic are special here?
Photo of Eric Pulvino

Eric Pulvino, Official Rep

  • 4,082 Points 4k badge 2x thumb
If you look at the /etc/cumulus/acl/policy.d/00control_plane.rules file you can see what the ports are for; 5342 is for CLAG (Multichassis Link Aggregation), and 10001 is for LNV (lightweight network virtualization -- which is the control protocol for exchanging VXlan VNID information):

cumulus@leaf1$ cat /etc/cumulus/acl/policy.d/00control_plane.rules
INGRESS_INTF = swp+

INGRESS_CHAIN = INPUT

INNFWD_CHAIN = INPUT,FORWARD

MARTIAN_SOURCES_4 = "240.0.0.0/5,127.0.0.0/8,224.0.0.0/8,255.255.255.255/32"

MARTIAN_SOURCES_6 = "ff00::/8,::/128,::ffff:0.0.0.0/96,::1/128"

CLAG_PORT = 5342

BFD_PORT = 3784

BFD_ECHO_PORT = 3785

BFD_MH_PORT = 4784

LNV_CTRL_PORT = 10001