python 64式: 第23式、分析oslo_messaging发送消息内部原理

最新推荐文章于 2022-05-21 20:03:07 发布

天地一扁舟

最新推荐文章于 2022-05-21 20:03:07 发布

阅读量754

点赞数

分类专栏： python 64式

本文链接：https://blog.csdn.net/qingyuanluofeng/article/details/90256573

版权

本文深入分析了OpenStack Ceilometer中oslo.messaging通知机制的工作原理，主要关注Ceilometer agent如何通过_notifier模块发送消息。通过调试代码，展示了从_polling_task的_send_notification方法到oslo.messaging.notify.notifier模块的sample方法的调用过程，再到最终的消息发送。详细剖析了参数、中间过程和涉及的关键参数，如event_type、payload等，为理解OpenStack中的监控数据传输提供了清晰的线索。

摘要由CSDN通过智能技术生成

分析: oslo_messaging.notifier.sample方法内部发送消息机制
1 代码总入口
ceilometer/agent/manager.py文件的PollingTask类的_send_notification方法
具体代码内容如下:
class PollingTask(object):
......
def _send_notification(self, samples):
self.manager.notifier.sample(
{},
'telemetry.polling',
{'samples': samples}
)
......

分析:
1) 查看参数:
(Pdb) p len(samples)

(Pdb) p samples
[{'counter_name': 'disk.write.requests', 'resource_id': u'3e8d82f9-8bbb-4b0f-9d7c-8420102d4ba7', 'timestamp': '2019-05-15T12:00:20.027386', 'counter_volume': 72L, 'user_id': u'385d7c49e4dc49da8370443ff5b2f3e9', 'message_signature': '40b5797508ad35506d720b1b7493c5eb87b15715762374b1df9ea05a80abe2e4', 'resource_metadata': {'status': u'active', 'ephemeral_gb': 0, 'disk_gb': 200, 'instance_host': u'node-2.domain.tld', 'kernel_id': None, 'image': {u'id': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', u'rel': u'bookmark'}], 'name': u'TestVM'}, 'ramdisk_id': None, 'host': u'85de2d0dfe5768ea397eec8b9af8021175a09d3f1673da5451502a84', 'device': ['vda'], 'flavor': {'name': u'1-1024-200', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/flavors/13', u'rel': u'bookmark'}], 'ram': 1024, 'ephemeral': 0, 'vcpus': 1, 'disk': 200, u'id': u'13'}, 'task_state': None, 'image_ref_url': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', 'memory_mb': 1024, 'root_gb': 200, 'display_name': u'new-4', 'name': u'instance-000000ee', 'vcpus': 1, 'instance_id': u'3e8d82f9-8bbb-4b0f-9d7c-8420102d4ba7', 'instance_type': u'1-1024-200', 'state': u'active', 'image_ref': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', 'OS-EXT-AZ:availability_zone': u'nova'}, 'source': 'openstack', 'counter_unit': 'request', 'project_id': u'5db2fbedf3b04193bd4c9a3705eb0e92', 'message_id': '0308a002-7709-11e9-9ff9-2e74ec40d945', 'monotonic_time': 4940322.617130146, 'counter_type': 'cumulative'}, {'counter_name': 'disk.write.requests', 'resource_id': u'58ed19e3-d424-4c0e-ba41-f1aa1b0babb1', 'timestamp': '2019-05-15T12:00:20.027386', 'counter_volume': 103L, 'user_id': u'385d7c49e4dc49da8370443ff5b2f3e9', 'message_signature': '7c4145364f56ae43b9cf3f35b38f72e091120118d379b5fb05a73c734341ff0b', 'resource_metadata': {'status': u'active', 'ephemeral_gb': 0, 'disk_gb': 20, 'instance_host': u'node-2.domain.tld', 'kernel_id': None, 'image': {u'id': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', u'rel': u'bookmark'}], 'name': u'TestVM'}, 'ramdisk_id': None, 'host': u'85de2d0dfe5768ea397eec8b9af8021175a09d3f1673da5451502a84', 'device': ['vda'], 'flavor': {'name': u'1-1024-20', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/flavors/8', u'rel': u'bookmark'}], 'ram': 1024, 'ephemeral': 0, 'vcpus': 1, 'disk': 20, u'id': u'8'}, 'task_state': None, 'image_ref_url': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', 'memory_mb': 1024, 'root_gb': 20, 'display_name': u'chaochao', 'name': u'instance-000000a6', 'vcpus': 1, 'instance_id': u'58ed19e3-d424-4c0e-ba41-f1aa1b0babb1', 'instance_type': u'1-1024-20', 'state': u'active', 'image_ref': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', 'OS-EXT-AZ:availability_zone': u'nova'}, 'source': 'openstack', 'counter_unit': 'request', 'project_id': u'5db2fbedf3b04193bd4c9a3705eb0e92', 'message_id': '03093ee0-7709-11e9-9ff9-2e74ec40d945', 'monotonic_time': 4940322.623567382, 'counter_type': 'cumulative'}]

(Pdb) p self.manager.notifier
<oslo_messaging.notify.notifier.Notifier object at 0x7f2c4876ded0>
(Pdb) p self.manager.notifier.__dict__
{'_serializer': <oslo_messaging.serializer.NoOpSerializer object at 0x7f2c4876df50>, '_driver_mgr': <stevedore.named.NamedExtensionManager object at 0x7f2c48771150>, 'retry': None, '_driver_names': ['messagingv2'], '_topics': ['notifications'], 'publisher_id': 'ceilometer.polling', 'transport': <oslo_messaging.transport.Transport object at 0x7f2c485df890>}

2) 调用oslo_messaging.notifier的sample方法

2 分析oslo_messaging.notifier的sample方法

> /usr/lib/python2.7/site-packages/oslo_messaging/notify/notifier.py(405)sample()
405 ->   def sample(self, ctxt, event_type, payload):
406     """Send a notification at sample level.
407
408     Sample notifications are for high-frequency events
409     that typically contain small payloads. eg: "CPU = 70%"
410
411     Not all drivers support the sample level
412     (log, for example) so these could be dropped.
413
414     :param ctxt: a request context dict
415     :type ctxt: dict
416     :param event_type: describes the event, for example
417     'compute.create_instance'
418     :type event_type: str
419     :param payload: the notification payload
420     :type payload: dict
421     :raises: MessageDeliveryFailure
422     """
423     self._notify(ctxt, event_type, payload, 'SAMPLE')

分析:
1) 输入参数
(Pdb) p ctxt
{}
(Pdb) p event_type
'telemetry.polling'
(Pdb) p payload
{'samples': [{'counter_name': 'disk.write.requests', 'resource_id': u'3e8d82f9-8bbb-4b0f-9d7c-8420102d4ba7', 'timestamp': '2019-05-15T12:00:20.027386', 'counter_volume': 72L, 'user_id': u'385d7c49e4dc49da8370443ff5b2f3e9', 'message_signature': '40b5797508ad35506d720b1b7493c5eb87b15715762374b1df9ea05a80abe2e4', 'resource_metadata': {'status': u'active', 'ephemeral_gb': 0, 'disk_gb': 200, 'instance_host': u'node-2.domain.tld', 'kernel_id': None, 'image': {u'id': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', u'rel': u'bookmark'}], 'name': u'TestVM'}, 'ramdisk_id': None, 'host': u'85de2d0dfe5768ea397eec8b9af8021175a09d3f1673da5451502a84', 'device': ['vda'], 'flavor': {'name': u'1-1024-200', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/flavors/13', u'rel': u'bookmark'}], 'ram': 1024, 'ephemeral': 0, 'vcpus': 1, 'disk': 200, u'id': u'13'}, 'task_state': None, 'image_ref_url': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', 'memory_mb': 1024, 'root_gb': 200, 'display_name': u'new-4', 'name': u'instance-000000ee', 'vcpus': 1, 'instance_id': u'3e8d82f9-8bbb-4b0f-9d7c-8420102d4ba7', 'instance_type': u'1-1024-200', 'state': u'active', 'image_ref': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', 'OS-EXT-AZ:availability_zone': u'nova'}, 'source': 'openstack', 'counter_unit': 'request', 'project_id': u'5db2fbedf3b04193bd4c9a3705eb0e92', 'message_id': '0308a002-7709-11e9-9ff9-2e74ec40d945', 'monotonic_time': 4940322.617130146, 'counter_type': 'cumulative'}, {'counter_name': 'disk.write.requests', 'resource_id': u'58ed19e3-d424-4c0e-ba41-f1aa1b0babb1', 'timestamp': '2019-05-15T12:00:20.027386', 'counter_volume': 103L, 'user_id': u'385d7c49e4dc49da8370443ff5b2f3e9', 'message_signature': '7c4145364f56ae43b9cf3f35b38f72e091120118d379b5fb05a73c734341ff0b', 'resource_metadata': {'status': u'active', 'ephemeral_gb': 0, 'disk_gb': 20, 'instance_host': u'node-2.domain.tld', 'kernel_id': None, 'image': {u'id': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', u'rel': u'bookmark'}], 'name': u'TestVM'}, 'ramdisk_id': None, 'host': u'85de2d0dfe5768ea397eec8b9af8021175a09d3f1673da5451502a84', 'device': ['vda'], 'flavor': {'name': u'1-1024-20', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/flavors/8', u'rel': u'bookmark'}], 'ram': 1024, 'ephemeral': 0, 'vcpus': 1, 'disk': 20, u'id': u'8'}, 'task_state': None, 'image_ref_url': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', 'memory_mb': 1024, 'root_gb': 20, 'display_name': u'chaochao', 'name': u'instance-000000a6', 'vcpus': 1, 'instance_id': u'58ed19e3-d424-4c0e-ba41-f1aa1b0babb1', 'instance_type': u'1-1024-20', 'state': u'active', 'image_ref': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', 'OS-EXT-AZ:availability_zone': u'nova'}, 'source': 'openstack', 'counter_unit': 'request', 'project_id': u'5db2fbedf3b04193bd4c9a3705eb0e92', 'message_id': '03093ee0-7709-11e9-9ff9-2e74ec40d945', 'monotonic_time': 4940322.623567382, 'counter_type': 'cumulative'}]}

2) 调用:
/usr/lib/python2.7/site-packages/oslo_messaging/notify/notifier.py(296)_notify()

3 分析_notify()方法
296 ->   def _notify(self, ctxt, event_type, payload, priority, publisher_id=None,
297     retry=None):
298     payload = self._serializer.serialize_entity(ctxt, payload)
299     ctxt = self._serializer.serialize_context(ctxt)
300
301     msg = dict(message_id=six.text_type(uuid.uuid4()),
302     publisher_id=publisher_id or self.publisher_id,
303     event_type=event_type,
304     priority=priority,
305     payload=payload,
306     timestamp=six.text_type(timeutils.utcnow()))
307
308     def do_notify(ext):
309     try:
310     ext.obj.notify(ctxt, msg, priority, retry or self.retry)
311     except Exception as e:
312     _LOG.exception(_LE("Problem '%(e)s' attempting to send to "
313     "notification system. Payload=%(payload)s"),
314     dict(e=e, payload=payload))
315
316     if self._driver_mgr.extensions:
317     self._driver_mgr.map(do_notify)

2)
(Pdb) p self._serializer
<oslo_messaging.serializer.NoOpSerializer object at 0x7f2c4876df50>
(Pdb) p payload
{'samples': [{'counter_name': 'disk.write.requests', 'resource_id': u'3e8d82f9-8bbb-4b0f-9d7c-8420102d4ba7', 'timestamp': '2019-05-15T12:00:20.027386', 'counter_volume': 72L, 'user_id': u'385d7c49e4dc49da8370443ff5b2f3e9', 'message_signature': '40b5797508ad35506d720b1b7493c5eb87b15715762374b1df9ea05a80abe2e4', 'resource_metadata': {'status': u'active', 'ephemeral_gb': 0, 'disk_gb': 200, 'instance_host': u'node-2.domain.tld', 'kernel_id': None, 'image': {u'id': u'a412bbcf-4cd1-438d-a095-60ac1123f25f', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', u'rel': u'bookmark'}], 'name': u'TestVM'}, 'ramdisk_id': None, 'host': u'85de2d0dfe5768ea397eec8b9af8021175a09d3f1673da5451502a84', 'device': ['vda'], 'flavor': {'name': u'1-1024-200', u'links': [{u'href': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/flavors/13', u'rel': u'bookmark'}], 'ram': 1024, 'ephemeral': 0, 'vcpus': 1, 'disk': 200, u'id': u'13'}, 'task_state': None, 'image_ref_url': u'http://nova-api.openstack.svc.cluster.local:8774/7cf89220dea34a1c8e30f7a1de2bb8a9/images/a412bbcf-4cd1-438d-a095-60ac1123f25f', 'memory_mb': 1024, 'root_gb': 200, 'display_name': u'new-4', 'name': u'instance-000000ee', 'vcpus