PX4模块设计之十八：Logger模块

1. Logger模块简介

结合Logger模块的命令帮助提示消息，做一个简单介绍。

功能特性：

1. 支持SD卡文件ULog数据保存
2. 支持MAVLink流ULog数据发送
3. 可同时支持SD卡和MAVLink流数据保存和发送
4. 支持双线程(任务)模式：a) ULog数据更新; b)ULog数据保存和发送
5. 支持双线程(任务)缓冲可配置满足SD卡和MAVLink链路性能调优

子命令：

• start：启动模块
• on：打开手动开启日志状态
• off：关闭手动开启日志状态 //只是手动开启日志的状态关闭，并不意味着日志不记录(模块内部是或的关系)
• stop：停止模块
• status：查询模块状态

start子命令，支持参数：

1. -m <val>：values: file|mavlink|all, default: all
2. -x：Enable/disable logging via Aux1 RC channel
3. -e：Enable logging right after start until disarm (otherwise only when armed)
4. -f：Log until shutdown (implies -e)
5. -t：Use date/time for naming log directories and files
6. -r <val>：Log rate in Hz, 0 means unlimited rate, default 280
7. -b <val>：Log buffer size in KiB, default 12
8. -p <val>：value: topic name, Poll on a topic instead of running with fixed rate (Log rate and topic intervals are ignored if this is set
9. -c <val>：Log rate factor (higher is faster), default 1.0

pxh> logger

### Description
System logger which logs a configurable set of uORB topics and system printf messages
(`PX4_WARN` and `PX4_ERR`) to ULog files. These can be used for system and flight performance evaluation,
tuning, replay and crash analysis.

It supports 2 backends:
- Files: write ULog files to the file system (SD card)
- MAVLink: stream ULog data via MAVLink to a client (the client must support this)

Both backends can be enabled and used at the same time.

The file backend supports 2 types of log files: full (the normal log) and a mission
log. The mission log is a reduced ulog file and can be used for example for geotagging or
vehicle management. It can be enabled and configured via SDLOG_MISSION parameter.
The normal log is always a superset of the mission log.

### Implementation
The implementation uses two threads:
- The main thread, running at a fixed rate (or polling on a topic if started with -p) and checking for
  data updates
- The writer thread, writing data to the file

In between there is a write buffer with configurable size (and another fixed-size buffer for
the mission log). It should be large to avoid dropouts.

### Examples
Typical usage to start logging immediately:
$ logger start -e -t

Or if already running:
$ logger on

Usage: logger <command> [arguments...]
 Commands:

   start
     [-m <val>]  Backend mode
                 values: file|mavlink|all, default: all
     [-x]        Enable/disable logging via Aux1 RC channel
     [-e]        Enable logging right after start until disarm (otherwise only when armed)
     [-f]        Log until shutdown (implies -e)
     [-t]        Use date/time for naming log directories and files
     [-r <val>]  Log rate in Hz, 0 means unlimited rate
                 default: 280
     [-b <val>]  Log buffer size in KiB
                 default: 12
     [-p <val>]  Poll on a topic instead of running with fixed rate (Log rate and topic intervals are ignored if this is set
                 values: <topic_name>
     [-c <val>]  Log rate factor (higher is faster)
                 default: 1.0

   on            start logging now, override arming (logger must be running)

   off           stop logging now, override arming (logger must be running)

   stop

   status        print status info

注：上述打印帮助来自函数Logger::print_usage，具体Coding这里不再赘述，有兴趣的同学可以独立深入。

2. 模块入口函数

2.1 主入口logger_main

logger_main
 ├──> int num = 1;
 ├──> <*(char *)&num != 1>   logger currently assumes little endian
 │   └──> PX4_ERR("Logger only works on little endian!\n");
 └──> return Logger::main(argc, argv);      // 这里调用了ModuleBase的main实现函数, 详见ModuleBase章节

注：具体入口后实现详见【PX4模块设计之十七：ModuleBase模块】

2.2 自定义子命令Logger::custom_command

Logger::custom_command
 ├──> <!is_running()>
 │   └──> return print_usage("logger not running");
 ├──> <subcommand=="on">
 │   └──> return get_instance()->set_arm_override(true);
 ├──> <subcommand=="off">
 │   └──> return get_instance()->set_arm_override(false);
 └──> return print_usage("unknown command");

2.3 日志主题uORB注册

该函数在px4_platform_init中调用，注册uORB日志主题，主要用于以下API日志记录（具体根据预编译决定）：

1. PX4_INFO --> __px4_log_modulename --> px4_log_modulename
2. PX4_PANIC --> __px4_log_modulename --> px4_log_modulename
3. PX4_ERR --> __px4_log_modulename --> px4_log_modulename
4. PX4_WARN --> __px4_log_modulename --> px4_log_modulename

px4_log_initialize
 ├──> log_message_s log_message{};
 ├──> log_message.severity = 6; // info
 ├──> strcpy((char *)log_message.text, "initialized uORB logging");
 ├──> log_message.timestamp = hrt_absolute_time();
 └──> orb_log_message_pub = orb_advertise_queue(ORB_ID(log_message), &log_message, log_message_s::ORB_QUEUE_LENGTH);

3. 重要实现函数

3.1 Logger::task_spawn

Logger::task_spawn
 ├──> _task_id = px4_task_spawn_cmd("logger",
 │				      SCHED_DEFAULT,
 │				      SCHED_PRIORITY_LOG_CAPTURE,
 │				      PX4_STACK_ADJUSTED(3700),
 │				      (px4_main_t)&run_trampoline,
 │				      (char *const *)argv);
 ├──> <_task_id < 0>
 │   ├──> _task_id = -1;
 │   └──> return -errno;
 └──> return OK

3.2 Logger::instantiate

Logger::instantiate
 ├──> <while ((ch = px4_getopt(argc, argv, "r:b:etfm:p:xc:", &myoptind, &myoptarg)) != EOF)>
 │   └──> [参数解析，异常参数直接返回失败]  
 ├──> Logger *logger = new Logger(backend, log_buffer_size, log_interval, poll_topic, log_mode, log_name_timestamp, rate_factor);  // 这里将前面解析的参数通过logger构造函数进行参数初始化。
 ├──> <logger == nullptr>
 │   └──> PX4_ERR("alloc failed");
 ├──> const char *logfile = getenv(px4::replay::ENV_FILENAME);
 └──> <logfile>
     └──> logger->setReplayFile(logfile);

这个函数在new一个logger对象的时候，大部分是参数赋值，唯一需要关注的是几个对象构造函数

1. ModuleParams(nullptr)
2. _event_subscription(ORB_ID::event)
3. _writer(backend, buffer_size)

3.2.1 Logger::Logger构造函数

Logger::Logger(LogWriter::Backend backend, size_t buffer_size, uint32_t log_interval, const char *poll_topic_name,
	       LogMode log_mode, bool log_name_timestamp, float rate_factor) :
	ModuleParams(nullptr),
	_log_mode(log_mode),
	_log_name_timestamp(log_name_timestamp),
	_event_subscription(ORB_ID::event),
	_writer(backend, buffer_size),
	_log_interval(log_interval),
	_rate_factor(rate_factor)
{
	if (poll_topic_name) {
		const orb_metadata *const *topics = orb_get_topics();

		for (size_t i = 0; i < orb_topics_count(); i++) {
			if (strcmp(poll_topic_name, topics[i]->o_name) == 0) {
				_polling_topic_meta = topics[i];
				break;
			}
		}

		if (!_polling_topic_meta) {
			PX4_ERR("Failed to find topic %s", poll_topic_name);
		}
	}
}

3.2.2 LogWriter::LogWriter构造函数

LogWriter::LogWriter(Backend configured_backend, size_t file_buffer_size)
	: _backend(configured_backend)
{
	if (configured_backend & BackendFile) {
		_log_writer_file_for_write = _log_writer_file = new LogWriterFile(file_buffer_size);

		if (!_log_writer_file) {
			PX4_ERR("LogWriterFile allocation failed");
		}
	}

	if (configured_backend & BackendMavlink) {
		_log_writer_mavlink_for_write = _log_writer_mavlink = new LogWriterMavlink();

		if (!_log_writer_mavlink) {
			PX4_ERR("LogWriterMavlink allocation failed");
		}
	}
}

3.2.3 LogWriter::LogWriter构造函数

这里体现了LogWriter可同时支持SD卡日志记录和MAVLink流数据通信。

LogWriter::LogWriter(Backend configured_backend, size_t file_buffer_size)
	: _backend(configured_backend)
{
	if (configured_backend & BackendFile) {
		_log_writer_file_for_write = _log_writer_file = new LogWriterFile(file_buffer_size);

		if (!_log_writer_file) {
			PX4_ERR("LogWriterFile allocation failed");
		}
	}

	if (configured_backend & BackendMavlink) {
		_log_writer_mavlink_for_write = _log_writer_mavlink = new LogWriterMavlink();

		if (!_log_writer_mavlink) {
			PX4_ERR("LogWriterMavlink allocation failed");
		}
	}
}

3.3 Logger::run

这个函数可是有点长，当前这个版本是379行代码。通常来说超长的函数是非常不容易理解的，而这些长函数通常又夹杂着全局变量，整体的耦合是非常难于理解的，也很难定位问题。所以设计先行的原则一定要时刻放在心上。这里估计是历史原因导致了这个问题，现在开源的要做重构可能也非常困难，我们就勉强看一下他的大致逻辑轮廓吧。

Logger::run
 ├──> <_writer.backend() & LogWriter::BackendFile> // 是否有SD卡日志记录要求
 │   ├──> int mkdir_ret = mkdir(LOG_ROOT[(int)LogType::Full], S_IRWXU | S_IRWXG | S_IRWXO);
 │   ├──> [创建失败，判断是否有MAVLink数据流，如果没有直接返回。无需进一步做日志记录]
 │   └──> [检查剩余空间，如果存储空间已满，直接返回]
 ├──> uORB::Subscription parameter_update_sub(ORB_ID(parameter_update));  // 订阅parameter_update
 ├──> <!initialize_topics()>
 │   └──> return  // 失败返回
 ├──> [根据定于topics情况，初始化最大max_msg_size]
 ├──> <!_writer.init()>  // LogWriter初始化
 │   └──> return  // 失败返回
 ├──> px4_register_shutdown_hook(&Logger::request_stop_static); // 注册模块优雅退出hook函数
 ├──> const bool disable_boot_logging = get_disable_boot_logging();
 ├──> <(_log_mode == LogMode::boot_until_disarm || _log_mode == LogMode::boot_until_shutdown) && !disable_boot_logging>
 │   └──> start_log_file(LogType::Full); 
 ├──> <_polling_topic_meta> // 命令行指定订阅topic
 │   └──> polling_topic_sub = orb_subscribe(_polling_topic_meta);
 │       └──> <polling_topic_sub < 0> 订阅失败
 │           └──> PX4_ERR("Failed to subscribe (%i)", errno);
 ├──> <!_polling_topic_meta> // 正常日志启动
 │   ├──> <_writer.backend() & LogWriter::BackendFile>  // 有SD卡文件日志记录情况
 │   │   ├──> const pid_t pid_self = getpid();const pthread_t writer_thread = _writer.thread_id_file();
 │   │   └──> watchdog_initialize(pid_self, writer_thread, timer_callback_data.watchdog_data);  // watchdog监控任务
 │   └──> hrt_call_every(&timer_call, _log_interval, _log_interval, timer_callback, &timer_callback_data);  // 启动高精度定时器
 ├──> <while (!should_exit())> 主任务循环
 │   ├──> const bool logging_started = start_stop_logging();  //Start/stop logging (depending on logging mode, by default when arming/disarming)
 │   ├──> handle_vehicle_command_update(); // check for logging command from MAVLink (start/stop streaming)
 │   ├──> <timer_callback_data.watchdog_triggered>
 │   │   ├──> timer_callback_data.watchdog_triggered = false;
 │   │   └──> initialize_load_output(PrintLoadReason::Watchdog);
 │   ├──> const hrt_abstime loop_time = hrt_absolute_time();
 │   ├──> [日志处理主流程，根据订阅uORB主题更新情况，将信息记录输出]
 │   ├──> update_params();
 │   ├──> <polling_topic_sub >= 0>
 │   │   └──> [订阅主题上poll]
 │   └──> <!(polling_topic_sub >= 0)>
 │       └──> while (px4_sem_wait(&timer_callback_data.semaphore) != 0) {}  //高精度定时等待信号量
 ├──> px4_lockstep_unregister_component(_lockstep_component);  // 清理退出模块
 ├──> stop_log_file(LogType::Full);
 ├──> stop_log_file(LogType::Mission);
 ├──> hrt_cancel(&timer_call);
 ├──> px4_sem_destroy(&timer_callback_data.semaphore);
 ├──> _writer.thread_stop();
 ├──> <polling_topic_sub >= 0>
 │   └──> orb_unsubscribe(polling_topic_sub);
 ├──> <_mavlink_log_pub>
 │   ├──> orb_unadvertise(_mavlink_log_pub);
 │   └──> _mavlink_log_pub = nullptr;
 └──> px4_unregister_shutdown_hook(&Logger::request_stop_static);

4. 总结

1. 日志模块作为模块是众多PX4基础模块应用的集中体现(除算法外的业务数据汇集点)。
2. 日志模块也是飞控黑匣子记录数据的主要来源。
3. 日志模块的记录数据也将为后续模拟复飞(replay)提供数据来源。

但是整体上涉及较多的业务环节，以及C/C++接口混用，目前还没有深入到记录消息以及ULog文件格式，时间间隔，记录速率，记录频率等内容。后续将会进一步深入研读和理解，另外以下订阅uORB消息成员变量是Logger类的私有成员，与日志记录开/关状态也有密切关系，比如：_manual_control_setpoint_sub/_vehicle_command_sub/_vehicle_status_sub。system printf messages (PX4_WARN and PX4_ERR)与_log_message_sub有关。其他还有mission/subscription等等。

	uORB::Subscription				_manual_control_setpoint_sub{ORB_ID(manual_control_setpoint)};
	uORB::Subscription				_vehicle_command_sub{ORB_ID(vehicle_command)};
	uORB::Subscription				_vehicle_status_sub{ORB_ID(vehicle_status)};
	uORB::SubscriptionInterval			_log_message_sub{ORB_ID(log_message), 20};
	uORB::SubscriptionInterval			_parameter_update_sub{ORB_ID(parameter_update), 1_s};

5. 参考资料

【1】PX4开源软件框架简明简介
【2】PX4 Logger模块
【3】PX4模块设计之二：uORB消息代理
【4】PX4模块设计之十二：High Resolution Timer设计
【5】PX4模块设计之十三：WorkQueue设计
【6】PX4模块设计之十四：Event设计
【7】PX4模块设计之十五：PX4 Log设计