Load XDP programs using the ip (iproute2) command

Load XDP programs using the ip (iproute2) command

What is XDP?

From the iovisor.org website:

XDP or eXpress Data Path provides a high performance, programmable network data path in the Linux kernel as part of the IO Visor Project. XDP provides bare metal packet processing at the lowest point in the software stack which makes it ideal for speed without compromising programmability. Furthermore, new functions can be implemented dynamically with the integrated fast path without kernel modification.

So XDP is a way to hook our eBPF programs very close to the network stack in order to do packets processing, encapsulation, de-encapsulation, get metrics etc..

The important thing you need to know is that you can write a program, that gets loaded into the kernel as if it was a module but that can be loaded without modifying it.

This kind program is called an eBPF program and is compiled to run against a special VM residing in the kernel that verifies and then executes those programs in a way that they cannot harm the running system.

You can look at the diagram below to visualize how the loading of eBPF programs works.

Said all of that, XDP programs are a specialized kind of eBPF programs with the additional capability to go lower level than kernel space by accessing driver space to act directly on packets.

So if we wanted to visualize the same diagram from an XDP point of view it will look like this.

In most cases, however your hardware may not support XDP so you will still be able to load XDP programs using the xdpgeneric driver so you will still get the improvements of doing this lower level but not as with a network card that supports offloading network processing to it instead of doing that stuff on your CPU(s).

What can I do with XDP?

This depends on how much imagination you have, some examples can be:

Monitoring of the network packet flow by populating a map back to the userspace, look at this example if you need inspiration;

Writing your own custom ingress/egress firewall;

Rewrite packet destinations, packets re-routing;

Packet inspection, security tools based on packets flowing;

Let’s try to load a program.

Requirements:

You will need a Linux machine with Kernel > 4.8 with clang (llvm ≥ 3.7) , iproute2 installed.

And the following NIC

“

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdpgeneric/id:29 qdisc pfifo_fast state UP group default qlen 1000

    altname enp4s0

    inet 192.168.121.164/24 brd 192.168.121.255 scope global dynamic noprefixroute eth0

       valid_lft 3560sec preferred_lft 3560sec

“

Step 1. Compile an XDP program using clang

Let’s take this XDP program as an example (there’s a more complete example later!)

“

#include <linux/bpf.h>

int main() {

  return XDP_DROP;

}

“

Compile it with:

$ clang -O2 -target bpf -c dropper.c -o dropper.o

Once loaded, this program drops every packet sent to the interface. If you change XDP_DROP with XDP_TX the packets will be sent back to the direction where they came from and you then change it to XDP_PASS, the packets will just continue flowing.

XDP_TX can also be used to transmit the packet to another destination if used after modifying the packet's destination address, mac address and TCP checksum for example.

Step 2: Load the dropper.o program

At the previous step, we compiled dropper.o from dropper.c. We can now use the ip command to load the program into the kernel.

# ip link set dev eth0 xdp obj dropper.o sec .text

The careful reader may have noticed that we are using the xdp flag in the previous command. That flag means that the kernel will do its best to load the XDP program on the network card as native. However not all the network cards support native XDP programs with hardware offloads, in that case, the kernel disables that.

Hardware offloads happen when the network driver is attached to a network card that can process the networking work defined in the XDP program so that the server’s CPU doesn’t have to do that.

In case you already know your card supports XDP you can use xdpdrv, if you know that It doesn’t you can use xdpgeneric .

Step 3: Test with traffic and unload

At this point, after loading the program you will notice that your tcpdump will stop receiving traffic because of the drop.

You can now stop the XDP program by unloading it

# ip link set dev eth0 xdp off

Step 4: Drop only UDP packets

For the very simple use case we had (drop everything) we didn’t need to access the xdp_md struct to get the context of the current packet flow, but now, since we want to selectively drop specific packets we need to do that.

In order to do that, however, we need to declare a function that has xdp_md *ctx as the first argument, to do that we need to use a different section than .text when loading our program, let’s see the program below:

#include <linux/bpf.h>

#include <linux/in.h>

#include <linux/if_ether.h>

#include <linux/ip.h>

#include <bpf/bpf_tracing.h>

#include <bpf/bpf_helper_defs.h>

 

#define SEC(NAME) __attribute__((section(NAME), used))

 

int dropper(struct xdp_md *ctx) {

        int ipsize = 0;

        void *data = (void *)(long)ctx->data;

        void *data_end = (void *)(long)ctx->data_end;

        struct ethhdr *eth = data;

        struct iphdr *ip;

 

        ipsize = sizeof(*eth);

        ip = data + ipsize;

        ipsize += sizeof(struct iphdr);

        if (data + ipsize > data_end) {

                return XDP_PASS;

        }

 

        if (ip->protocol == IPPROTO_UDP) {

                char fmt[] = "ipsize %d, src:%x, dst:%x\n";

 

                bpf_trace_printk(fmt, sizeof(fmt), ipsize, ip->saddr, ip->daddr);

 

                return XDP_DROP;

        }

 

        return XDP_PASS;

}

 

char _license[] SEC("license") = "GPL";

There’s some boilerplate to extract the Ethernet frame from which we can then extract the information related to the packet like the protocol in this case that we use to check the protocol and drop all the UDP packets

Let’s compile it!

$ clang -O2 -target bpf -c samples/bpf/udp.c -o udp.o -I tools/lib

And now we can verify the symbol table with objdump to see if our section is there.

# objdump -t udp.o

 

udp.o:     file format elf64-little

 

SYMBOL TABLE:

0000000000000000 l    df *ABS*  0000000000000000 udp.c

00000000000000d8 l       .text  0000000000000000 LBB0_3

0000000000000000 l    d  .text  0000000000000000 .text

0000000000000000 g     O license        0000000000000004 _license

0000000000000000 g     F .text  00000000000000e8 dropper

Cool, let’s load the function using the section .text always on the same interface we used before eth0.

# ip link set dev eth0 xdp obj udp.o sec .text

Now let us look at the output.

 

# cat /sys/kernel/debug/tracing/trace

# tracer: nop

#

# entries-in-buffer/entries-written: 105/5925335   #P:6

#

#                              _-----=> irqs-off

#                             / _----=> need-resched

#                            | / _---=> hardirq/softirq

#                            || / _--=> preempt-depth

#                            ||| /     delay

#           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION

#              | |       |   ||||       |         |

<idle>-0  [005] d.s5 100852.332321: bpf_trace_printk: ipsize 34, src:b379a8c0, dst:a479a8c0

 

<idle>-0  [005] d.s5 100852.400320: bpf_trace_printk: ipsize 34, src:b379a8c0, dst:a479a8c0

 

<idle>-0 [005] d.s5 100852.400338: bpf_trace_printk: ipsize 34, src:b379a8c0, dst:a479a8c0

 

<idle>-0 [005] d.s5 100852.400349: bpf_trace_printk: ipsize 34, src:b379a8c0, dst:a479a8c0

…

References

https://github.com/iovisor/bpf-docs

https://www.youtube.com/watch?v=JRFNIKUROPE

https://www.kernel.org/doc/html/latest/networking/af_xdp.html

https://cilium.readthedocs.io/en/latest/bpf/

https://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html?highlight=XDP

https://www.netdevconf.org/2.1/slides/apr6/bertin_Netdev-XDP.pdf