Bandwidth Management/Rate Limiting

  • 1
  • Question
  • Updated 2 years ago
  • (Edited)
I'm certainly no networking expert so forgive any stupidity here. If I first explain my topology:

I currently have a 10Gb fiber connection into a Cisco 6500. I then have two fiber cables from the Cisco 6500 to each of the Edge Core Cumulus Switches for redundancy and load balancing.

Each has a connection to a Radius/DHCP server. 
Each also has a connection to an Ubiquiti Power Beam acting as a host antenna. It is then wirelessly linked to a client Antenna to provide them internet access.

This is the most simple form of the setup I require.

The Cisco 6500 is a temporary device used to do the BGP until such time as this has been configured on the Edge Cores to replace this. It is transparent and simply passes the traffic through.

Bandwidth/Rate Limiting is currently something I am struggling to grasp. From the following :

I understand I can use Layer 2 to limit the rate of the connection, I presume from this, I can, using the MAC address specify a specific rate. From the link above however from what I understand this can only be achieved using Netfilter and is only applicable to ingress traffic (I presume what would be the customer upload).
I cannot see any way that limiting the egress tarffic is currently possible (I presume again this would effectively be the client download speed).

Looking at the follwing link:

Particulary the last comment by hunterthomson, it appears it may be possible to limit egress traffic on Layer3 using iptables and traffic control. If I understand correctly I can set some static rules in the firewall (based on the client IP) to limit the egress rate. Providing I give the same IP to the same client this should work.

My Radius/DHCP server will be configured to provide static IP's.

So to wrap it all together, potentially I can limit ingress traffic using Layer 2 and Netfilter by referencing the users MAC. I can then on Layer 3 tag traffic using iptables and traffic control so that when the request is fullfilled and sent back to the Edgecore it limits the rate at which this traffic is allowed to pass through to the client.

If this is the case I assume I could create a quick script and cron to run every few minutes, collect all the DHCP leases from my Radius/DHCP including the necessary speed. Using this information I could then dynamically create both Netfilter and Iptable rules which would achieve dynamic filtering of both ingress and egress traffic.

My question really boils down to whether this is a feasible solution, whether any one has tried this or whether I have the wrong end of the stick completely.

Any other information or discussion about the possibiltiy of this is greatly appreciated.

Just for my info I found the following which may help

root@leaf2:/home/cumulus# cat /etc/cumulus/acl/policy.d/customer1.rules
-A $INGRESS_CHAIN --in-interface $INGRESS_INTF -j police --set-mode KB --set-rate 100 --set-burst 100
Photo of Jon H

Jon H

  • 92 Points 75 badge 2x thumb

Posted 2 years ago

  • 1
Photo of Eric Pulvino

Eric Pulvino, Official Rep

  • 3,868 Points 3k badge 2x thumb
I'm having a hard time understanding your goal here. If you're trying to rate-limit traffic out of certain ports as it moves through the switch that is possible. If you're trying to rate limit the amount of traffic each host can send based on mac address or IP address that would technically be possible in IPv4 although I haven't tested it personally.
Photo of Alan Dobinson

Alan Dobinson

  • 102 Points 100 badge 2x thumb
Hi Eric,

Many thanks for the reply.

It is more a mixture of the two.

In short, the users have an unlimited connection in terms of total bandwidth, however,  the rate of traffic is what needs to be limited

Effectively if I have say 100 users going via swp1 and out to the internet (if we say swp1 is bridged with swp2 that connects to the internet), I need to individually either by MAC or IP limit the incoming and outgoing rate a specific user can have.

This would also be better if we could apply this to a group. For example of the 100 users I have 50 on 10Mb Download and a 1Mb upload and I have a further 50 on a 20Mb download and a 2Mb upload, I would effectively need to limit the users to these speeds.

I have been told with netfilter is is possible to do this but only the ingress not the egress. So I am trying to see how this can be achieved. 

Can you point me in the direction of the tools you believe would help em achieve my goal if the above is possible.
Photo of Eric Pulvino

Eric Pulvino, Official Rep

  • 3,868 Points 3k badge 2x thumb
This configuration is not typical. You're essentially trying to configure per IP policing in both directions. The number of rules required to do this does not scale to large numbers of hosts. It is one of those scenarios where it sounds easy but is technically complex to implement and as a result most do not implement scenarios like this at a per-user level, instead they limit bandwidth at a more global level i.e. limiting the whole pool of bandwidth to the entire subnet of users.

In Cumulus the configuration you're talking about would look something like this....

root@leaf2:mgmt-vrf:/etc/cumulus/acl/policy.d# cat ./20pfp.rules
#Testing for
-A FORWARD -d -j POLICE --set-mode KB --set-rate 20480 --set-burst 100
-A FORWARD -s -j POLICE --set-mode KB --set-rate 2048 --set-burst 100

root@leaf2:mgmt-vrf:/etc/cumulus/acl/policy.d# sudo cl-acltool -L all
Listing rules of type iptables:

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination

0 0 POLICE all -- any any anywhere POLICE mode:KB rate:20480 burst:100
0 0 POLICE all -- any any anywhere POLICE mode:KB rate:2048 burst:100

Of course that is the required rules for one host.

Most folks would to this at the port or subnet level (below I show what the rule might look like for the entire subnet of hosts):

root@leaf2:mgmt-vrf:/etc/cumulus/acl/policy.d# cat ./20pfp.rules
#Testing for
-A FORWARD -d -j POLICE --set-mode KB --set-rate 512000 --set-burst 1000
-A FORWARD -s -j POLICE --set-mode KB --set-rate 51200 --set-burst 1000