Trace Events (UserspaceAPI)
We’re in userspacenow, so you can leave the kernel source directory, but you do need to be rootand you may need to mount a filesystem. This is because some distributions(like Ubuntu) have an allergy to debugging (seriously, they even disable thingslike gdb -p).
Try and change intothe /sys/kernel/debug/tracingdirectory.
# cd /sys/kernel/debug/tracing
If this fails, you’llneed to mount the debugfsfilesystem and try again.
# mount -t debugfs none /sys/kernel/debug
# cd /sys/kernel/debug/tracing
With that done, weshould make sure tracing is enabled.
# cat tracing_enabled
1
If that’s 0, enableit:
# echo 1 > tracing_enabled
So we’ve enabledtracing, but what can we trace? Trace events are exposed in the eventssub-directory in two levels, the first is the subsystem and the second are thetrace events themselves. Since in my last blog post we were looking at tracing forks, it would begreat if there were trace events for doing just that. This is where it helps tobe able to git greparound the kernel source and recognise trace events, so you at least know theright subsystem name; and it turns out that the sched subsystem has exactly the events wewanted.
deathspank tracing# ls events/sched
enable sched_process_exit/ sched_stat_sleep/
filter sched_process_fork/ sched_stat_wait/
sched_kthread_stop/ sched_process_free/ sched_switch/
sched_kthread_stop_ret/ sched_process_wait/ sched_wait_task/
sched_migrate_task/ sched_stat_iowait/ sched_wakeup/
sched_pi_setprio/ sched_stat_runtime/ sched_wakeup_new/
sched_process_fork sounds exactly right, if you look at it, it’s a directory thatcontains four files: enable,filter, format and id. I bet you canguess how to enable fork tracing, but if not:
# cat events/sched/sched_process_fork/enable
0
# echo 1 > events/sched/sched_process_fork/enable
Pretty painless, sogo ahead and run a few things, and turn the tracing off again when you’re done.
# echo 0 > events/sched/sched_process_fork/enable
Now let’s look at theresult of our trace; recall that every trace event comes with a free printk() of formattedoutput? We can find the output from those in the top-level trace file.
# tracer: nop
#
# TASK-PID CPU# TIMESTAMP FUNCTION
# | | | | |
zsh-2667 [001] 6658.716936: sched_process_fork: comm=zsh pid=2667 child_comm=zsh child_pid=2748
So for each processfork, we get the parent and child process ids along with the process name.Pretty much exactly what we want!