Tc: ingress policing and ifb mirroring

Question:

I'm trying to setup traffic shaping on a Linux gateway as written here. The script needs to be customized because I have multiple LAN interfaces. So to shape the LAN side I am planning to create a ifb pseudo device like so:

     modprobe ifb
     ip link set dev ifb0 up
    /sbin/tc qdisc add dev $WAN_INTERFACE ingress
    /sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0

The script from the gist repo mentioned above has these lines:

 /sbin/tc qdisc add dev $WAN_INTERFACE handle ffff: ingress
    /sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip sport $INTERACTIVE_PORT 0xffff flowid :1
    /sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 1 u32 match ip dport $INTERACTIVE_PORT 0xffff flowid :1
    /sbin/tc filter add dev $WAN_INTERFACE parent ffff: protocol ip prio 5 0 u32 match ip src 0.0.0.0/0 police rate $MAX_DOWNRATE_INGRESS burst 20k drop flowid :2

This code and the ifb interface creation code don't get on well together. The customized script gets executed, but ifb0 device doesn't show any traffic stats. If I comment out ingress gist repo code (quoted above), then ifb0 device shows the number of packets that are transferred. Also these lines cannot be executed together:

/sbin/tc qdisc add dev $WAN_INTERFACE ingress
/sbin/tc qdisc add dev $WAN_INTERFACE handle ffff: ingress

I get file exists error. So, how can I shape ingress on WAN_INTERFACE and at the same time also shape the traffic that goes to LAN via ifb0 device?

Answers:

IFB is an alternative to tc filters for handling ingress traffic, by redirecting it to a virtual interface and treat is as egress traffic there.You need one ifb interface per physical interface, to redirect ingress traffic from eth0 to ifb0, eth1 to ifb1 and so on.

When inserting the ifb module, tell it the number of virtual interfaces you need. The default is 2:

modprobe ifb numifbs=1

Now, enable all ifb interfaces:

ip link set dev ifb0 up # repeat for ifb1, ifb2, ...

And redirect ingress traffic from the physical interfaces to corresponding ifb interface. For eth0 -> ifb0:

tc qdisc add dev eth0 handle ffff: ingress
tc filter add dev eth0 parent ffff: protocol ip u32 match u32 0 0 action mirred egress redirect dev ifb0

Again, repeat for eth1 -> ifb1, eth2 -> ifb2 and so on, until all the interfaces you want to shape are covered.

Now, you can apply all the rules you want. Egress rules for eth0 go as usual in eth0. Let's limit bandwidth, for example:

tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:1 htb rate 1mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 1mbit

Needless to say, repeat for eth1, eth2, ...

Ingress rules for eth0, now go as egress rules on ifb0 (whatever goes into ifb0 must come out, and only eth0 ingress traffic goes into ifb0). Again, a bandwidth limit example:

tc qdisc add dev ifb0 root handle 1: htb default 10
tc class add dev ifb0 parent 1: classid 1:1 htb rate 1mbit
tc class add dev ifb0 parent 1:1 classid 1:10 htb rate 1mbit

The advantage of this approach is that egress rules are much more flexible than ingress filters. Filters only allow you to drop packets, not introduce wait times, for example. By handling ingress traffic as egress you can setup queue disciplines, with traffic classes and, if need be, filters. You get access to the whole tc tree, not only simple filters.

traffic shaping - Tc: ingress policing and ifb mirroring - Server Fault