Ingress qdisc
All qdiscs discussed so far are egress qdiscs. Each interface however can also have an ingress qdisc which is not used to send packets out to the network adaptor. Instead, it allows you to apply tc filters to packets coming in over the interface, regardless of whether they have a local destination or are to be forwarded.
As the tc filters contain a full Token Bucket Filter implementation, and are also able to match on the kernel flow estimator, there is a lot of functionality available. This effectively allows you to police incoming traffic, before it even enters the IP stack.
14.4.1. Parameters & usage
The ingress qdisc itself does not require any parameters. It differs from other qdiscs in that it does not occupy the root of a device. Attach it like this:
# delete original
tc qdisc del dev eth0 ingress
tc qdisc del dev eth0 root
All qdiscs discussed so far are egress qdiscs. Each interface however can also have an ingress qdisc which is not used to send packets out to the network adaptor. Instead, it allows you to apply tc filters to packets coming in over the interface, regardless of whether they have a local destination or are to be forwarded.
As the tc filters contain a full Token Bucket Filter implementation, and are also able to match on the kernel flow estimator, there is a lot of functionality available. This effectively allows you to police incoming traffic, before it even enters the IP stack.
14.4.1. Parameters & usage
The ingress qdisc itself does not require any parameters. It differs from other qdiscs in that it does not occupy the root of a device. Attach it like this:
# delete original
tc qdisc del dev eth0 ingress
tc qdisc del dev eth0 root
# add new qdisc and filter
tc qdisc add dev eth0 ingress
tc filter add dev eth0 parent ffff: protocol ip prio 50 u32 match ip src 0.0.0.0/0 police rate 2048kbps burst 1m drop flowid :1
tc qdisc add dev eth0 root tbf rate 2048kbps latency 50ms burst 1m
I played a bit with the ingress qdisc after seeing Patrick and Stef
talking about it and came up with a few notes and a few questions.
: The ingress qdisc itself has no parameters. The only thing you can do
: is using the policers. I have a link with a patch to extend this :
: http://www.cyberus.ca/~hadi/patches/action/ Maybe this can help.
:
: I have some more info about ingress in my mail files, but I have to
: sort it out and put it somewhere on docum.org. But I still didn't
: found the the time to do so.
Regarding policers and the ingress qdisc. I have never used them before
today, but have the following understanding.
About the ingress qdisc:
- ingress qdisc (known as "ffff:") can't have any children classes (hence the existence of IMQ)
- the only thing you can do with the ingress qdisc is attach filters
About filtering on the ingress qdisc:
- since there are no classes to which to direct the packets, the only reasonable option (reasonable, indeed!) is to drop the packets
- with clever use of filtering, you can limit particular traffic signatures to particular uses of your bandwidth
Here's an example of using an ingress policer to limit inbound traffic
from a particular set of IPs on a per IP basis. In this case, traffic
from each of these source IPs is limited to a T1's worth of bandwidth.
Note that this means that this host can receive up to 1536kbit (768kbit +
768kbit) worth of bandwidth from these two source IPs alone.
# -- start of script
#! /bin/ash
#
# -- simulate a much smaller amount of bandwidth than the 100MBit interface
#
RATE=1536kbit
DEV=eth0
SOURCES="10.168.53.2/32 10.168.73.10/32 10.168.28.20/32"
# -- attach our ingress qdisc
#
tc qdisc add dev $DEV ingress
# -- cap bandwidth from particular source IPs
#
for SOURCE in $SOURCES ; do
tc filter add dev $DEV parent ffff: protocol ip \
u32 match ip src $SOURCE flowid :1 \
police rate $RATE mtu 12k burst 10k drop
done
# -- end of script
Now, if you are using multiple public IPs on your masquerading/SNAT host,
you can use "u32 match ip dst $PER_IP" with a drop action to force a
particular rate on inbound traffic to that IP.
My entirely unquantified impression is that latency suffers as a result,
but traffic is indeed bandwidth limited.
Just a few notes of dissection:
tc filter add dev $DEV # -- the usual beginnings
parent ffff: # -- the ingress qdisc itself
protocol ip # -- more preamble | make sure to visit
u32 match ip # -- u32 classifier | http://lartc.org/howto/
src $SOURCE # -- could also be "dst $SOME_LOCAL_IP"
flowid :1 # -- ??? (but it doesn't work without this)
police rate $RATE # -- put a policer here
mtu 12k burst 10k # -- ???
drop # -- drop packets exceeding our police params
Maybe a guru or two out there (Stef?, Bert?, Jamal?, Werner?) can explain
why mtu needs to be larger than 1k (didn't work for me anyway) and also
how these other parameters should be used.
talking about it and came up with a few notes and a few questions.
: The ingress qdisc itself has no parameters. The only thing you can do
: is using the policers. I have a link with a patch to extend this :
: http://www.cyberus.ca/~hadi/patches/action/ Maybe this can help.
:
: I have some more info about ingress in my mail files, but I have to
: sort it out and put it somewhere on docum.org. But I still didn't
: found the the time to do so.
Regarding policers and the ingress qdisc. I have never used them before
today, but have the following understanding.
About the ingress qdisc:
- ingress qdisc (known as "ffff:") can't have any children classes (hence the existence of IMQ)
- the only thing you can do with the ingress qdisc is attach filters
About filtering on the ingress qdisc:
- since there are no classes to which to direct the packets, the only reasonable option (reasonable, indeed!) is to drop the packets
- with clever use of filtering, you can limit particular traffic signatures to particular uses of your bandwidth
Here's an example of using an ingress policer to limit inbound traffic
from a particular set of IPs on a per IP basis. In this case, traffic
from each of these source IPs is limited to a T1's worth of bandwidth.
Note that this means that this host can receive up to 1536kbit (768kbit +
768kbit) worth of bandwidth from these two source IPs alone.
# -- start of script
#! /bin/ash
#
# -- simulate a much smaller amount of bandwidth than the 100MBit interface
#
RATE=1536kbit
DEV=eth0
SOURCES="10.168.53.2/32 10.168.73.10/32 10.168.28.20/32"
# -- attach our ingress qdisc
#
tc qdisc add dev $DEV ingress
# -- cap bandwidth from particular source IPs
#
for SOURCE in $SOURCES ; do
tc filter add dev $DEV parent ffff: protocol ip \
u32 match ip src $SOURCE flowid :1 \
police rate $RATE mtu 12k burst 10k drop
done
# -- end of script
Now, if you are using multiple public IPs on your masquerading/SNAT host,
you can use "u32 match ip dst $PER_IP" with a drop action to force a
particular rate on inbound traffic to that IP.
My entirely unquantified impression is that latency suffers as a result,
but traffic is indeed bandwidth limited.
Just a few notes of dissection:
tc filter add dev $DEV # -- the usual beginnings
parent ffff: # -- the ingress qdisc itself
protocol ip # -- more preamble | make sure to visit
u32 match ip # -- u32 classifier | http://lartc.org/howto/
src $SOURCE # -- could also be "dst $SOME_LOCAL_IP"
flowid :1 # -- ??? (but it doesn't work without this)
police rate $RATE # -- put a policer here
mtu 12k burst 10k # -- ???
drop # -- drop packets exceeding our police params
Maybe a guru or two out there (Stef?, Bert?, Jamal?, Werner?) can explain
why mtu needs to be larger than 1k (didn't work for me anyway) and also
how these other parameters should be used.