iptables connection tracking for the control plane?

  • 2
  • Question
  • Updated 11 months ago
I understand that many iptables features (connection tracking, etc...) are not supported by the underlying switching hardware on switches running Cumulus Linux.

It feels like I should still be able to use these features to protect the control plane, but I'm finding that cl-acltool is refusing to install them.

Perhaps there's a different way (something other than /etc/cumulus/acl/policy.d) to get these rules installed?

Maybe I'm confused and these features can't be used, even on the INPUT/OUTPUT chains?

How else are people protecting (filtering, not CoPP) TCP listeners exposed by the control plane? I see that sshd is compiled with libwrap, but I'd feel better with something that's universally applicable (bgpd does not have libwrap) and runs a bit lower in the stack.

Thanks!
Photo of Chris Marget

Chris Marget

  • 162 Points 100 badge 2x thumb

Posted 1 year ago

  • 2
Photo of Sean Cavanaugh

Sean Cavanaugh, Alum

  • 3,380 Points 3k badge 2x thumb
I would like to preface this with that Cumulus Linux is acting as designed.  Connection oriented iptables, also known as firewall zones, is not function of data center switches and every vendor using similar ASICs has the same restriction.  Cumulus Linux is not a replacement for a firewall in the data center, but does have some powerful features (which you are using with cl-acltool -i) to augment firewalls.
It feels like I should still be able to use these features to protect the control plane, but I'm finding that cl-acltool is refusing to install them.
It can't install something to hardware that is not possible.  The ASIC is not capable of doing some of the features you are talking about at line-speed.  Technically you "could" hack it to work on eth0 (which is a software port) and use NAT, conntrack whatever you wanted on eth0, but your eth0 throughput is going to be severely less than 1Gbps and pretty useless for anything a data center switch should be pushing.  Even if you installed it directly with iptables (not recommended btw) you would wipe them off everytime cl-acltool -i rules were installed.  This is generally a bad idea and every GSS engineer will scratch their head trying to figure out what is going on if a ticket is opened up on this device.  Does that make sense?  I can elaborate here further if needed, but think of the control plane as an aircraft tower, and the data plane as all the planes going around.  Its telling the hardware what to do, but not actually flying himself.

Maybe I'm confused and these features can't be used, even on the INPUT/OUTPUT chains?
Again, technically they could, but not on a hardware port.  The CPU of data center switches is considerably more powerful then they have been in the past, but there is no way a DC switch can even handle 40Gbps of traffic blasted at it.  The hardware will drop this before it would ever reach the CPU.  This is by design.

How else are people protecting (filtering, not CoPP) TCP listeners exposed by the control plane? 

TCP listeners on the switch itself, or servers connected to it?  It is unclear was the intended goal is here.  If the goal is protecting the control plane of the switch you will see all the CoPP policies you referenced to before.  Many people block SSH from their in-band network entirely.  The only major TCP application running on the switch I can think of is BGP, and BGP has authentication mechanisms on it.  I am at a loss on how this isn't protected if i am policing at a hardware level for BGP connections, and authenticating the connections as they come in.

 I see that sshd is compiled with libwrap, but I'd feel better with something that's universally applicable (bgpd does not have libwrap) and runs a bit lower in the stack.
Can you elaborate here, I am a bit confused in the question.
(Edited)
Photo of Chris Marget

Chris Marget

  • 162 Points 100 badge 2x thumb
Hi Sean,

I'm interested exclusively in protecting TCP services running on the switch.
bgp, ssh, clagd... That sort of thing.

I have a requirement for in-band management, so am interested in options that apply to traffic which ingress via swpX and is destined for the kernel.

If the answer is "this can't be done because ASIC", then I'll accept it and use stateless filters here.

But I confess, I don't understand why this is the answer. I don't intend for the policy to be applied by the ASIC (even though the traffic enters there), but rather by the kernel (where it's headed).

My logic here: A normal server isn't installing these policies on the ingress hardware (intel/broadcom/mellanox NIC), but in the kernel. So why would the "to/from this box" policies I desire be limited by the capabilities of the ingress hardware?

Transit traffic is a whole different story of course. I understand why that can't work.

Thanks!
Photo of Sean Cavanaugh

Sean Cavanaugh, Alum

  • 3,380 Points 3k badge 2x thumb
But I confess, I don't understand why this is the answer. I don't intend for the policy to be applied by the ASIC (even though the traffic enters there), but rather by the kernel (where it's headed).
So if you are intended target is the switch, you could use rules on the INPUT, OUTPUT chain but you would need to use iptables directly rather than cl-acltool.  I have never seen this in production.

Traffic-> ASIC -> Kernel / CPU

So usually a customer will ask "why can't we nat?" and the reason is if we go to the CPU we can process enough data at the line-rate of the port.  If you are just trying to run an application or tracking it will work.... its just that the cl-acltool is not really setup to care about that, b/c it cares about the ASIC state and what the ASIC can do.  If you have a rule that BLOCKS/DROPs its droped before it ever reaches the kernel/CPU vs what you are talking about.  I have not tried it but if you apply the rules directly, the destination is the switch this will act just like a server.  
Photo of Chris Marget

Chris Marget

  • 162 Points 100 badge 2x thumb
Thanks, Sean.

If I understand you correctly, my scheme should work, but there's no cumulus-provided framework for implementing it.

It sounds like I just need to think about how badly I want this capability. It seems like good hygiene, but I don't want to create an unsupportable environment.

I totally get that transit traffic should never hit the CPU and capabilities there are limited to the set of things implemented by both the ASIC and switchd.

Is this the right time to ask for a --control-plane-only or [iptables-control-plane] feature in cl-acltool?
Photo of Sean Cavanaugh

Sean Cavanaugh, Alum

  • 3,380 Points 3k badge 2x thumb
So they have control plane feature with NCLU that programs the iptables rules for you with tab complete.  The problem is that it does not do some of the rules I think you want.  I think your feature request needs to be more specific with what you want to do (just post the rules or something).  We can work with your region's SE to get your feature request heard/reviewed by engineering to see it makes sense.  You can start a thread with me if you don't know who your SE is, just email me at sean@ and we can start a conversation there.

For NCLU, you make a rule like this->

cumulus@c1l1:mgmt-vrf:~$ net add acl ipv4 TEST
accept : Allow, permit, etc
drop : Deny, drop, etc
dscp : Differentiated services
erspan : Extended Switched Port Analyzer
police : Restrict traffic rates
priority : Priority number
set-class : Set class values
set-qos : Set QOS values
span : Switched Port Analyzer
table : Table


and apply it to control plane like this
cumulus@c1l1:mgmt-vrf:~$ net add control-plane
acl : access-list

then do a "net commit"
(Edited)
Photo of Chris Marget

Chris Marget

  • 162 Points 100 badge 2x thumb
Ahh, cool! I'll look into this, thanks!

I know who my SE is, but I'm (a) a two-switch customer who's (b) running on a demo license. I'll get a bit more aggressive with Pete L. when the license/support PO finally goes through :)

/chris
Photo of Eric Pulvino

Eric Pulvino, Official Rep

  • 4,082 Points 4k badge 2x thumb
Pete thrives with aggression. You don't need to wait for a PO to get aggressive with Pete :)
Photo of Sylvain Munaut

Sylvain Munaut

  • 746 Points 500 badge 2x thumb
I'd definitely like to be able to combine boht HW filtering and OS/sw filtering for the INPUT chain, I wrote about my thought for a better iptables mapping for this there :

https://getsatisfaction.cumulusnetworks.com/cumulus/topics/better-hw-mapping-for-input-acl-chain-how...