How to priotize packets using tc and cgroups

Question

I am trying to prioritize the packets that are generated from a certain process group so that they'll be picked first to be transmitted from the PC. I aim to do this by using cgroups and tc, but it seems not to work.

First I set up cgroups on ubuntu as follows,

modprobe cls_cgroup # load this module to get net_cls

mkdir /sys/fs/cgroup/net_cls  # mount point

mount -t cgroup net_cls -onet_cls /sys/fs/cgroup/net_cls/

mkdir /sys/fs/cgroup/net_cls/foo # new cgroup

echo 0x00010001 > /sys/fs/cgroup/foo/net_cls.classid  # echo in a class id

echo 2348 > /sys/fs/cgroup/net_cls/foo/tasks # echo in pid of firefox

tc qdisc add dev eth0 root handle 1: pri

tc qdisc add dev eth0 parent 1:1 handle 10: sfq

tc qdisc add dev eth0 parent 1:2 handle 20: sfq

tc qdisc add dev eth0 parent 1:3 handle 30: sfq

and after browsing in firefox and running,

tc -s qdisc ls dev eth0

I get,

qdisc prio 1: root refcnt 2 bands 3 priomap  1 2 2 2 1 2 0 0 1 1 1 1 1 1 1 1
 Sent 29351 bytes 154 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc sfq 10: parent 1:1 limit 127p quantum 1514b divisor 1024 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc sfq 20: parent 1:2 limit 127p quantum 1514b divisor 1024 
 Sent 27873 bytes 143 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 
qdisc sfq 30: parent 1:3 limit 127p quantum 1514b divisor 1024 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0 

Instead I want the traffic to flow in handle 10, what am I doing wrong?

Answer 1

The correct way to do this requires informing tc that you are to be using cgroups. This has been verified on Ubuntu 12.04 with a 3.10 kernel.

Ensure you have net_cls support

$ cat /proc/cgroups 
#subsys_name    hierarchy   num_cgroups enabled
cpuset  1   2   1
cpu 1   2   1
cpuacct 1   2   1
memory  1   2   1
net_cls 1   2   1
blkio   1   2   1

if not,

Compile a kernel with net_cls support

Just put all these options in your .config. These appear not to exist in menuconfig.

CONFIG_NET_CLS=y
CONFIG_NET_CLS_BASIC=m
CONFIG_NET_CLS_TCINDEX=m
CONFIG_NET_CLS_ROUTE4=m
CONFIG_NET_CLS_FW=m
CONFIG_NET_CLS_U32=m
CONFIG_NET_CLS_RSVP=m
CONFIG_NET_CLS_RSVP6=m
CONFIG_NET_CLS_FLOW=m
CONFIG_NET_CLS_CGROUP=y
CONFIG_NET_CLS_ACT=y
CONFIG_NET_CLS_IND=y

then make and install.

Ensure you have an /etc/fstab entry

# echo "cgroup /sys/fs/cgroup cgroup defaults 0 0" >> /etc/fstab
# reboot

Create the test cgroup and set it up

Some cgroup setups complain with generic errors if cpuset is not set. You also must convert your major and minor tc class ID into hex of 0xAAAABBBB where AAAA is major and BBBB is minor.

# mkdir /sys/fs/cgroup/clstest
# /bin/echo 0 > /sys/fs/cgroup/clstest/cpuset.mems
# /bin/echo 0 > /sys/fs/cgroup/clstest/cpuset.cpus
# /bin/echo 0x100001 > /sys/fs/cgroup/clstest/net_cls.classid

Configure tc

# tc qdisc add dev eth2 root handle 10: htb
# tc class add dev eth2 parent 10: classid 10:1 htb rate 10mbit
# tc filter add dev eth2 parent 10: protocol ip prio 10 handle 1: cgroup

Echo tasks into cgroup

(but only one at a time)

# echo $FIREFOX_PID > /sys/fs/cgroup/clstest/tasks

Modify tc class

# tc class change dev eth2 parent 10: classid 10:1 htb rate 40mbit

EDIT:

I have been unable to make this work with ingress. Only egress (upload) appears to be working. tc does not appear to take the cgroup option with ingress.

Answer 2

You should add class and filter to the traffic you want to control (assume you want to control HTTP traffic).

# tc class add dev eth0 parent 1:1 classid 1:1 htb rate 500mbit ceil 800mbit burst 10k prio 10
# tc filter add dev eth0 parent 1:1 protocol ip prio 10 u32 match ip dport 80 0xffff flowid 1:1

Then you can use iptraf to verify the connection rate.