Load XDP programs using the ip (iproute2) command
What is XDP?
From the iovisor.org website:
XDP or eXpress Data Path provides a high performance, programmable network data path in the Linux kernel as part of the IO Visor Project. XDP provides bare metal packet processing at the lowest point in the software stack which makes it ideal for speed without compromising programmability. Furthermore, new functions can be implemented dynamically with the integrated fast path without kernel modification.
So XDP is a way to hook our eBPF programs very close to the network stack in order to do packets processing, encapsulation, de-encapsulation, get metrics etc..
The important thing you need to know is that you can write a program, that gets loaded into the kernel as if it was a module but that can be loaded without modifying it.
This kind program is called an eBPF program and is compiled to run against a special VM residing in the kernel that verifies and then executes those programs in a way that they cannot harm the running system.
You can look at the diagram below to visualize how the loading of eBPF programs works.
Said all of that, XDP programs are a specialized kind of eBPF programs with the additional capability to go lower level than kernel space by accessing driver space to act directly on packets.
So if we wanted to visualize the same diagram from an XDP point of view it will look like this.
In most cases, however your hardware may not support XDP so you will still be able to load XDP programs using the xdpgeneric driver so you will still get the improvements of doing this lower level but not as with a network card that supports offloading network processing to it instead of doing that stuff on your CPU(s).
What can I do with XDP?
This depends on how much imagination you have, some examples can be:
Monitoring of the network packet flow by populating a map back to the userspace, look at this example if you need inspiration;
Writing your own custom ingress/egress firewall;
Rewrite packet destinations, packets re-routing;
Packet inspection, security tools based on packets flowing;
Let’s try to load a program.
Requirements:
You will need a Linux machine with Kernel > 4.8 with clang (llvm ≥ 3.7) , iproute2 installed.
And the following NIC
“
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 xdpgeneric/id:29 qdisc pfifo_fast state UP group default qlen 1000
altname enp4s0
inet 192.168.121.164/24 brd 192.168.121.255 scope global dynamic noprefixroute eth0
valid_lft 3560sec preferred_lft 3560sec
“
Step 1. Compile an XDP program using clang
Let’s take this XDP program as an example (there’s a more complete example later!)
“
#include <linux/bpf.h>
int main() {
return XDP_DROP;
}
“
Compile it with:
$ clang -O2 -target bpf -c dropper.c -o dropper.o
Once loaded, this program drops every packet sent to the interface. If you change XDP_DROP with XDP_TX the packets will be sent back to the direction where they came from and you then change it to XDP_PASS, the packets will just continue flowing.
XDP_TX can also be used to transmit the packet to another destination if used after modifying the packet's destination address, mac address and TCP checksum for example.
Step 2: Load the dropper.o program
At the previous step, we compiled dropper.o from dropper.c. We can now use the ip command to load the program into the kernel.
# ip link set dev eth0 xdp obj dropper.o sec .text
The careful reader may have noticed that we are using the xdp flag in the previous command. That flag means that the kernel will do its best to load the XDP program on the network card as native. However not all the network cards support native XDP programs with hardware offloads, in that case, the kernel disables that.
Hardware offloads happen when the network driver is attached to a network card that can process the networking work defined in the XDP program so that the server’s CPU doesn’t have to do that.
In case you already know your card supports XDP you can use xdpdrv, if you know that It doesn’t you can use xdpgeneric .
Step 3: Test with traffic and unload
At this point, after loading the program you will notice that your tcpdump will stop receiving traffic because of the drop.
You can now stop the XDP program by unloading it
# ip link set dev eth0 xdp off
Step 4: Drop only UDP packets
For the very simple use case we had (drop everything) we didn’t need to access the xdp_md struct to get the context of the current packet flow, but now, since we want to selectively drop specific packets we need to do that.
In order to do that, however, we need to declare a function that has xdp_md *ctx as the first argument, to do that we need to use a different section than .text when loading our program, let’s see the program below:
#include <linux/bpf.h>
#include <linux/in.h>
#include <linux/if_ether.h>
#include <linux/ip.h>
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_helper_defs.h>
#define SEC(NAME) __attribute__((section(NAME), used))
int dropper(struct xdp_md *ctx) {
int ipsize = 0;
void *data = (void *)(long)ctx->data;
void *data_end = (void *)(long)ctx->data_end;
struct ethhdr *eth = data;
struct iphdr *ip;
ipsize = sizeof(*eth);
ip = data + ipsize;
ipsize += sizeof(struct iphdr);
if (data + ipsize > data_end) {
return XDP_PASS;
}
if (ip->protocol == IPPROTO_UDP) {
char fmt[] = "ipsize %d, src:%x, dst:%x\n";
bpf_trace_printk(fmt, sizeof(fmt), ipsize, ip->saddr, ip->daddr);
return XDP_DROP;
}
return XDP_PASS;
}
char _license[] SEC("license") = "GPL";
There’s some boilerplate to extract the Ethernet frame from which we can then extract the information related to the packet like the protocol in this case that we use to check the protocol and drop all the UDP packets
Let’s compile it!
$ clang -O2 -target bpf -c samples/bpf/udp.c -o udp.o -I tools/lib
And now we can verify the symbol table with objdump to see if our section is there.
# objdump -t udp.o
udp.o: file format elf64-little
SYMBOL TABLE:
0000000000000000 l df *ABS* 0000000000000000 udp.c
00000000000000d8 l .text 0000000000000000 LBB0_3
0000000000000000 l d .text 0000000000000000 .text
0000000000000000 g O license 0000000000000004 _license
0000000000000000 g F .text 00000000000000e8 dropper
Cool, let’s load the function using the section .text always on the same interface we used before eth0.
# ip link set dev eth0 xdp obj udp.o sec .text
Now let us look at the output.
# cat /sys/kernel/debug/tracing/trace
# tracer: nop
#
# entries-in-buffer/entries-written: 105/5925335 #P:6
#
# _-----=> irqs-off
# / _----=> need-resched
# | / _---=> hardirq/softirq
# || / _--=> preempt-depth
# ||| / delay
# TASK-PID CPU# |||| TIMESTAMP FUNCTION
# | | | |||| | |
<idle>-0 [005] d.s5 100852.332321: bpf_trace_printk: ipsize 34, src:b379a8c0, dst:a479a8c0
<idle>-0 [005] d.s5 100852.400320: bpf_trace_printk: ipsize 34, src:b379a8c0, dst:a479a8c0
<idle>-0 [005] d.s5 100852.400338: bpf_trace_printk: ipsize 34, src:b379a8c0, dst:a479a8c0
<idle>-0 [005] d.s5 100852.400349: bpf_trace_printk: ipsize 34, src:b379a8c0, dst:a479a8c0
…
References
https://github.com/iovisor/bpf-docs
https://www.youtube.com/watch?v=JRFNIKUROPE
https://www.kernel.org/doc/html/latest/networking/af_xdp.html
https://cilium.readthedocs.io/en/latest/bpf/
https://suricata.readthedocs.io/en/latest/capture-hardware/ebpf-xdp.html?highlight=XDP
https://www.netdevconf.org/2.1/slides/apr6/bertin_Netdev-XDP.pdf