Recently met a ROS node crash issue which just shows below information
xxx_node-3] process has died [pid 32584, exit code -11, cmd/home/robert/Documents/test_ws/src/flight
-xxx_build/devel/lib/flight_xxx_xxx/solar_xxx_node
_name: =solar node
_log:=/home/r
obert/.ros/log/0f839167-4411-11ed-92be-48b02d3d8078/solar_orchestrator_node-3.logl
log file: /home/robert/.ros/log/0f839167-4411-11ed-92be-48b02d3d8078/solar_xxx_node-3*.
To debug this issue, we need core dump to see more crash information, here is the steps:
1. Check and set the core size to unlimited
$ ulimit -a
core file size (blocks, -c) 0 # <-- Prevents core dumps
Default corefile size is 0, it means it will not generate coredump file
Set the core size to unlimited:
ulimit -c unlimited
Then again ulimit -a
$ ulimit -a
core file size (blocks, -c) unlimited
core file size is changed as unlimited
2. Check if coredump file generation with pid
$ cat /proc/sys/kernel/core_uses_pid
0
0 means core dump file pid is deactivated. let’s activate
echo 1 | sudo tee /proc/sys/kernel/core_uses_pid ...
$ cat /proc/sys/kernel/core_uses_pid
1
Now, it can generate coredump with pid
3. Run the ros node and reproduce the crash,such as below
(base) test@xavier-rd-x:~/bumbleTest_one$ roslaunch test test.launch
~~~
[module-1] process has died [pid 10821, exit code -11, cmd
~~~
The PID number is 8779