之前整理了常见的几种配置文件的读写操作,能够处理大多数额应用场景。最近遇到了protobuf的配置,因为这种配置中节点key有重复的问题,无法使用之前的几种处理方式进行读写。
查看了很多网上文章,没有找到将protobuf配置按照文本方式读写的介绍,更没有将配置读取成常用json或者dict格式进行处理的。
如果你也用到了protobuf配置文件,也想把他当做dict进行读取、修改。你可以参考一下我整理的代码,希望能帮到你。
from logging import Logger
from google.protobuf import text_format
from google.protobuf import json_format
logger = Logger('DrsuOperate').getlog()
class ProtobufFIle():
# protobuf文件处理
def __init__(self, conf_file, message_obj):
"""
初始化
:param conf_file: 需要解析的protobuf文件路径
:param message_obj: protobuf对应的proto message对象
"""
self.conf_file = conf_file
self.message_obj = message_obj
self.message = None
def read_conf(self):
try:
with open(self.conf_file, "r", encoding='utf-8') as fp:
text = fp.read()
self.message = text_format.Parse(text, self.message_obj)
logger.info('读取配置文件 {} 成功'.format(self.conf_file))
except Exception as e:
logger.error('文件读取失败: {}'.format(str(e)))
return False
return True
def write_conf(self):
try:
with open(self.conf_file, "w", encoding='utf-8') as fp:
fp.write(text_format.MessageToString(self.message))
logger.info('写入配置文件 {} 成功'.format(self.conf_file))
except Exception as e:
logger.error('文件写入失败: {}'.format(str(e)))
return False
return True
def convert_to_dict(self):
if not self.message:
self.read_conf()
try:
message_dict = json_format.MessageToDict(self.message)
logger.info('配置转换为字典格式成功 {}'.format(message_dict))
except Exception as e:
logger.error('配置文件转换为字典格式失败: {}'.format(str(e)))
return False
return message_dict
def convert_from_dict(self, dict_obj):
self.message = json_format.ParseDict(dict_obj, self.message_obj)
if __name__ == '__main__':
dag_camera = r'\dag_camera.config'
dag_camera_msg = dag_camera_protobuf_pb2.DagCamera()
protobuf_read = ProtobufFIle(dag_camera, dag_camera_msg)
protobuf_read.convert_to_dict()
eg: 以一个protobuf文件的读取为例:
- 需要解析的protobuf文件文件名为dag_camera_protobuf.txt,内容:
subnode_config { subnodes { id: 2 name: "MultiCameraProcessSubnode" reserve: "device_id_0:CameraFrontRight;onboard:1;" process: "type:MultiCameraTl;gpu_id:0;pb_obj:1;is_visualization:0;" type: SUBNODE_IN } subnodes { id: 4 name: "MonocularCameraProcessSubnode" reserve: "device_id:CameraFrontLeft;onboard:1;" process: "type:Object;gpu_id:0;pb_obj:1;is_visualization:0;obstacle_detector:camera_obstacle_detector_centernet_v3" type: SUBNODE_IN } }
- 与protobuf对应的proto文件名为:dag_camera_protobuf,内容:
syntax = "proto3"; message SubnodeConfig { enum SubnodeType { SUBNODE_OUT = 0; SUBNODE_IN = 1; } message Subnode { int32 id = 1; string name = 2; string reserve = 3; string process = 4; SubnodeType type = 5; } repeated Subnode subnodes = 1; } message DagCamera { repeated SubnodeConfig subnode_config = 1; }
- proto文件生成的python脚本的命令:
protoc --python_out=./dag_camera_protobuf
,会生成一个名为dag_camera_protobuf_pb2.py的python文件。 - 使用上面的解析方法解析protobuf文件
import dag_camera_protobuf_pb2 import ProtobufFIle dag_camera_file = 'dag_camera_protobuf.txt' dag_camera_msg = dag_camera_protobuf_pb2.DagCamera() protobuf_read = ProtobufFIle(dag_camera_file, dag_camera_msg) protobuf_read.convert_to_dict()
附:
6. proto的环境配置:pip install protobuf
7. proto规范,可以参考连接https://www.kaifaxueyuan.com/basic/protobuf3/default-values.html
8. protoc的下载地址https://repo1.maven.org/maven2/com/google/protobuf/protoc/