vrf and mgmt-vrf packages from Cumulus Networks

  • 1
  • Question
  • Updated 5 days ago
In reading through your tutorial related to VRF's (http://schd.ws/hosted_files/ossna2017/fe/vrf-tutorial-oss.pdf) I've come to a point where I am having troubles getting services on a vrf and note that the article mentions vrf & mgmt-vrf packages from Cumulus & includes a link (https://github.com/CumulusNetworks/vrf) however there are not configuration files such that we could just pull the git repo and run 'configure', 'make' and then 'make install'.
Does that exist elsewhere/can you point me to other documentation that might get me past this point?
I am specifically to a point of switch services to @mgmt that fail due to missing files/directories (which are all auto-generated in Cumulus) thus I believe I'm missing something that is in the referenced package. This is on an Ubuntu host: 16.04 on 4.14 kernel with updated iproute2 and the ifupdown2 package that the above referenced tutorial suggested. 
Photo of Troy Kau

Troy Kau

  • 240 Points 100 badge 2x thumb

Posted 3 weeks ago

  • 1
Photo of David Ahern

David Ahern, Employee

  • 282 Points 250 badge 2x thumb
hi Troy: At the moment that code base is just text files. You can run 'make install' to install files, 'make rpm' to create an rpm package to install or 'dpkg-buildpackage -uc -us' to build a deb package to install. I will add a README with that information as well as a 'make deb' target for the debian package.
(Edited)
Photo of Troy Kau

Troy Kau

  • 240 Points 100 badge 2x thumb
thanks for the quick response David, I'll keep an eye on git for the update & give it a try
Photo of Troy Kau

Troy Kau

  • 240 Points 100 badge 2x thumb
David: We were able to get several things working but I'm still stumbling on a few items. I can't seem to get rsyslog nor snmp@ working. 
On the rsyslog I've added a directive specifying "Device=mgmt" but I'm getting an omfwd error.
On snmp I'm getting the following logs whenever I poll the device externally (we can run an snmpwalk & get responses locally but nothing off the box): "snmpd[3673]: send response: Failure in sendto"
I added a forwarding rule and got rsyslog sending via the vrf but prior to that it seems it was unable to connect. Does this sound familiar/should I need to add the forwarding rule to the FIB?
Have you seen this snmp behavior before/have any suggestions?

Thanks in advance.
Photo of David Ahern

David Ahern, Employee

  • 282 Points 250 badge 2x thumb
Troy: Looks like my last response was eaten by goblins, so I'll try again.

rsyslog needs to be version 8.24 or higher. Ubuntu 16.04 has an older version.

net-snmp I need to check. At one point we needed a patched net-snmp to avoid it adding IP_PKTINFO with an ifindex of 0 which essentially removes the vrf binding done by the vrf command.  That patch has been reverted and I tested net-snmp in Cumulus Linux yesterday and it worked fine with mgmt vrf.
Photo of Troy Kau

Troy Kau

  • 240 Points 100 badge 2x thumb
I loaded rsyslog 8.32 (v8stable repo latest) and now I'm getting:
 rsyslogd: No UDP socket could successfully be initialized, some functionality may be disabled.  [v8.32.0]
 rsyslogd: create UDP socket bound to device failed: Operation not permitted [v8.32.0]

This is with the following in the rsyslog configuration:
$template GRAYLOGRFC5424,"<%PRI%>%PROTOCOL-VERSION% %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n"
action( type="omfwd" Target="10.100.21.30" Device="mgmt" Port="1514" Protocol="udp" template="GRAYLOGRFC5424" )
Photo of David Ahern

David Ahern, Employee

  • 282 Points 250 badge 2x thumb
Hmmm.... sounds like rsyslog is not running as root. That is the only way to get EPERM denied. Do this:
strace -o /tmp/rsyslogd.trace -fF -tt -T /usr/sbin/rsyslogd -n

Take a look at the trace file see which operation specifically is failing.
Photo of Troy Kau

Troy Kau

  • 240 Points 100 badge 2x thumb
nice catch - it was running as syslog; running as root it seems to be just fine.  Thanks
now I just need to get the snmp part - is there a specific release you would recommend?
Photo of David Ahern

David Ahern, Employee

  • 282 Points 250 badge 2x thumb
snmpd release is not clear. I can see that the Ubuntu 16.04 version definitely has the sendmsg / IP_PKTINFO bug and that is why you get the sendto failures. I'll need to find some time to compare upstream code to the Cumulus version and see what change has it working. Perhaps early next week.