[问题篇]VMWare搭建Openstack——Cinder创建扩展Volume的状态Error的问题

原创 2015年07月06日 16:30:28
这两天安装Cinder for icehouse,安装过程比较顺利,但是安装完毕之后,在创建volume和查看cinder list时,状态为error.
sm@controller:~$ cinder list
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
|                  ID                  | Status | Display Name | Size | Volume Type | Bootable |             Attached to              |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+
| fe7f56d7-a319-4f67-b821-4904922889c7 | in-use |  myVolume1   |  4   |     None    |  false   | 614a2641-1e8d-4442-9704-6ab62e3f39d5 |
+--------------------------------------+--------+--------------+------+-------------+----------+--------------------------------------+

由于该问题,已经解决,目前查看到的cinder list状态是in-use,目前我曾经遇到过的关于创建volume的状态(status)有四个:

1、正常创建,还没有使用,状态为:Available

2、正常创建,已经使用挂接,状态为:in-use

3、创建失败,状态为:Error

4、创建失败,状态为:Creating。

对于Creating,有可能是你创建的Volume过大,可能等一些时间,但是我说的这个创建1GB的,也会出现一种处于Creating状态,这肯定是有问题的。


对于状态为Error时,我查看了控制节点的:sudo cat /var/log/cinder/cinder-scheduler.log日志

我看到了两个比较典型的错误,No Valid Host was Found

2015-06-12 23:10:15.319 10092 INFO oslo.messaging._drivers.impl_rabbit [req-202316a3-6086-41be-9608-92b52349f7ff - - - - -] Connected to AMQP server on 192.168.3.180:5672
2015-06-12 23:10:15.478 10092 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.3.180:5672
2015-06-12 23:17:46.610 10092 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'eca4528a0b5b46fd990b07ec5bee364a', 'tenant': u'b36c6f68c7bc45eab07b0b62ddc516a8', 'user_identity': u'eca4528a0b5b46fd990b07ec5bee364a b36c6f68c7bc45eab07b0b62ddc516a8 - - -'}
2015-06-12 23:17:46.694 10092 ERROR cinder.scheduler.flows.create_volume [req-00fd9195-c835-4bb1-9cf8-37a24d1700f1 eca4528a0b5b46fd990b07ec5bee364a b36c6f68c7bc45eab07b0b62ddc516a8 - - -] Failed to schedule_create_volume: No valid host was found.
还有一种,提示volume service is down or disabled
2015-06-13 17:47:21.637 1060 AUDIT cinder.service [-] Starting cinder-scheduler node (version 2014.1.3)
2015-06-13 17:47:21.723 1060 ERROR oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] AMQP server on 192.168.3.180:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds.
2015-06-13 17:47:22.769 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Reconnecting to AMQP server on 192.168.3.180:5672
2015-06-13 17:47:22.769 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Delaying reconnect for 1.0 seconds...
2015-06-13 17:47:23.783 1060 ERROR oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] AMQP server on 192.168.3.180:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 3 seconds.
2015-06-13 17:47:26.787 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Reconnecting to AMQP server on 192.168.3.180:5672
2015-06-13 17:47:26.787 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Delaying reconnect for 1.0 seconds...
2015-06-13 17:47:27.807 1060 ERROR oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] AMQP server on 192.168.3.180:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 5 seconds.
2015-06-14 16:25:25.340 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Reconnecting to AMQP server on 192.168.3.180:5672
2015-06-14 16:25:25.341 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Delaying reconnect for 1.0 seconds...
2015-06-14 16:25:26.350 1060 INFO oslo.messaging._drivers.impl_rabbit [req-c7a22eb6-7396-4c1b-99de-a51d90d9e502 - - - - -] Connected to AMQP server on 192.168.3.180:5672
2015-06-14 16:25:26.855 1060 INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.3.180:5672
2015-06-14 16:25:29.661 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:26:08.255 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:27:08.259 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:28:08.258 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:29:08.249 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': None, 'tenant': None, 'user_identity': u'- - - - -'}
2015-06-14 16:30:04.562 1060 WARNING cinder.context [-] Arguments dropped when creating context: {'user': u'5bcdce96f7e749fbb157842c9eaf7765', 'tenant': u'7a9a86dfd657482a859688fb97e2375f', 'user_identity': u'5bcdce96f7e749fbb157842c9eaf7765 7a9a86dfd657482a859688fb97e2375f - - -'}
2015-06-14 16:30:04.653 1060 WARNING cinder.scheduler.host_manager [req-dcd56f67-84ab-4bab-ade1-614c6c06df16 5bcdce96f7e749fbb157842c9eaf7765 7a9a86dfd657482a859688fb97e2375f - - -] volume service is down or disabled. (host: cinder)

先稍微说一我的OpenStack实验架构,一个控制节点,一个网络节点,一个计算节点,一个存储(Cinder)节点

也就是说我的存储节点的volume服务是有问题的,所以找不到存储节点。那么重启存储节点的

service cinder-volume restart
问题仍然没有解决,这个让我非常纳闷,为什么明明服务启动没有问题,还是找不到存储节点呢?


经过上午搜索,看到人家博客介绍了又是以前我在Neutron碰到的问题,NTP的时间同步问题,当时间同步之后,问题解决。

sm@controller:~$ cinder service-list
+------------------+------------+------+---------+-------+----------------------------+
|      Binary      |    Host    | Zone |  Status | State |         Updated_at         |
+------------------+------------+------+---------+-------+----------------------------+
| cinder-scheduler | controller | nova | enabled |   up  | 2015-06-14T08:46:58.000000 |
|  cinder-volume   |   cinder   | nova | enabled |  down | 2015-06-14T08:33:18.000000 |
|  cinder-volume   |   cinder   | nova | enabled |   up  | 2015-06-14T08:46:52.000000 |
+------------------+------------+------+---------+-------+----------------------------+


但是又看到了新问题,我看到了两个一样的Cinder-volumn,我明明只有一个存储节点,这个好处理,直接进入数据库,将信息删除即可。

sm@controller:~$  sudo mysql -uroot -p#db4smtest#
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 52
Server version: 5.5.40-0ubuntu0.14.04.1 (Ubuntu)

Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> use cinder
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> show tables;
+--------------------------+
| Tables_in_cinder         |
+--------------------------+
| backups                  |
| encryption               |
| iscsi_targets            |
| migrate_version          |
| quality_of_service_specs |
| quota_classes            |
| quota_usages             |
| quotas                   |
| reservations             |
| services                 |
| snapshot_metadata        |
| snapshots                |
| transfers                |
| volume_admin_metadata    |
| volume_glance_metadata   |
| volume_metadata          |
| volume_type_extra_specs  |
| volume_types             |
| volumes                  |
+--------------------------+
19 rows in set (0.00 sec)

mysql> desc services
    -> ;
+-------------------+--------------+------+-----+---------+----------------+
| Field             | Type         | Null | Key | Default | Extra          |
+-------------------+--------------+------+-----+---------+----------------+
| created_at        | datetime     | YES  |     | NULL    |                |
| updated_at        | datetime     | YES  |     | NULL    |                |
| deleted_at        | datetime     | YES  |     | NULL    |                |
| deleted           | tinyint(1)   | YES  |     | NULL    |                |
| id                | int(11)      | NO   | PRI | NULL    | auto_increment |
| host              | varchar(255) | YES  |     | NULL    |                |
| binary            | varchar(255) | YES  |     | NULL    |                |
| topic             | varchar(255) | YES  |     | NULL    |                |
| report_count      | int(11)      | NO   |     | NULL    |                |
| disabled          | tinyint(1)   | YES  |     | NULL    |                |
| availability_zone | varchar(255) | YES  |     | NULL    |                |
| disabled_reason   | varchar(255) | YES  |     | NULL    |                |
+-------------------+--------------+------+-----+---------+----------------+
12 rows in set (0.00 sec)

mysql> select host from services;
+------------+
| host       |
+------------+
| controller |
| cinder     |
| cinder     |
+------------+
3 rows in set (0.00 sec)

mysql> select host,id from services;
+------------+----+
| host       | id |
+------------+----+
| controller |  1 |
| cinder     |  2 |
| cinder     |  3 |
+------------+----+
3 rows in set (0.00 sec)

mysql> select host,id,updated_at from services;
+------------+----+---------------------+
| host       | id | updated_at          |
+------------+----+---------------------+
| controller |  1 | 2015-06-14 08:49:28 |
| cinder     |  2 | 2015-06-14 08:49:32 |
| cinder     |  3 | 2015-06-14 08:33:18 |
+------------+----+---------------------+
3 rows in set (0.00 sec)

mysql> delete from services where id=3;
Query OK, 1 row affected (0.01 sec)

这时候,在查看Cinder的服务信息

sm@controller:~$ source admin-openrc.sh
sm@controller:~$ cinder service-list
+------------------+------------+------+---------+-------+----------------------------+
|      Binary      |    Host    | Zone |  Status | State |         Updated_at         |
+------------------+------------+------+---------+-------+----------------------------+
| cinder-scheduler | controller | nova | enabled |   up  | 2015-06-14T13:31:37.000000 |
|  cinder-volume   |   cinder   | nova | enabled |   up  | 2015-06-14T13:31:43.000000 |
+------------------+------------+------+---------+-------+----------------------------+

这时候,再创建Volume时,状态就可以了.


期间还碰到了一个问题:使用cinder create命令创建volume时,这个volume一直处于creating的状态,不论多长时间。

解决方法:仍然在数据库中删除,问题原因不详。

有可能是创建volume时出错了又没有及时返回,导致一直处于creating状态,对于这种卷,只能到数据库里去删除了,也不用删除数据库的整条记录,只改两个字段,先设置status=deleted,然后再设置deleted=1就可以了,这个volume就被删除了。

mysql> select id,status,deleted from volumes;
+--------------------------------------+----------+---------+
| id                                   | status   | deleted |
+--------------------------------------+----------+---------+
| 125e2d2d-0e96-48cc-874a-1b92c95c2cae | deleted  |       1 |
| 378f83a4-c72a-4ef8-aaae-557ce851146e | deleted  |       1 |
| 50ae05fa-9e89-4512-84d5-64bb041b2d05 | deleted  |       1 |
| 6c6d0513-8ca2-4fe6-9a9f-f412ae9e8cd9 | deleted  |       1 |
| 7064d4cc-743b-4b02-9882-9af64b189e38 | deleted  |       1 |
| 787d6ea0-fbb2-4e93-a8fa-11ca36503fdf | deleted  |       1 |
| cfcffb39-ac03-4beb-af2e-eb6c1d698c33 | deleted  |       1 |
| d0de1dab-2166-49c6-b3a2-965d28b61d59 | deleted  |       1 |
| df92f31b-d1e5-42c4-86fb-02bdfb4d9858 | creating |       0 |
| f20bf6d1-cad1-42f8-9258-eac9cae273ac | deleted  |       1 |
| f8ad5cdc-be86-40a8-9af1-99db1589c2e8 | deleted  |       1 |
| fbf69bbc-5af5-41b4-a260-43c0ef1de5d9 | deleted  |       1 |
+--------------------------------------+----------+---------+
12 rows in set (0.00 sec)

mysql> update volumes set status='deleted',deleted=1 where id=
    -> 'df92f31b-d1e5-42c4-86fb-02bdfb4d9858';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select id,status,deleted from volumes;
+--------------------------------------+---------+---------+
| id                                   | status  | deleted |
+--------------------------------------+---------+---------+
| 125e2d2d-0e96-48cc-874a-1b92c95c2cae | deleted |       1 |
| 378f83a4-c72a-4ef8-aaae-557ce851146e | deleted |       1 |
| 50ae05fa-9e89-4512-84d5-64bb041b2d05 | deleted |       1 |
| 6c6d0513-8ca2-4fe6-9a9f-f412ae9e8cd9 | deleted |       1 |
| 7064d4cc-743b-4b02-9882-9af64b189e38 | deleted |       1 |
| 787d6ea0-fbb2-4e93-a8fa-11ca36503fdf | deleted |       1 |
| cfcffb39-ac03-4beb-af2e-eb6c1d698c33 | deleted |       1 |
| d0de1dab-2166-49c6-b3a2-965d28b61d59 | deleted |       1 |
| df92f31b-d1e5-42c4-86fb-02bdfb4d9858 | deleted |       1 |
| f20bf6d1-cad1-42f8-9258-eac9cae273ac | deleted |       1 |
| f8ad5cdc-be86-40a8-9af1-99db1589c2e8 | deleted |       1 |
| fbf69bbc-5af5-41b4-a260-43c0ef1de5d9 | deleted |       1 |
+--------------------------------------+---------+---------+
12 rows in set (0.00 sec)

mysql> exit

版权声明: 专注于"GIS+"前沿技术的研究与交流,将云计算技术、大数据技术、容器技术、物联网与GIS进行深度融合,探讨"GIS+"技术和行业解决方案;文章允许转载,但必须以链接方式注明源地址,否则追究法律责任!

相关文章推荐

devstack 环境中cinder-volume 服务无法启动的问题

Command: sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings -o name stack-...

别以为真懂Openstack: 虚拟机创建的50个步骤和100个知识点

转载自:http://blog.itpub.net/18796236/viewspace-1840119/ 别以为真懂Openstack!先别着急骂我,我也没有说我真懂Openstack...

Openstack Cinder创建卷错误排错过程

由于项目中使用的OpenStack一直停留在F版本,因此在操作系统(Ubuntu 12.04 LTS)不断更新的情况下,难免会出现各种莫名其妙的问题。 安装完Openstack,各子模块均运行正常,开...

深入块存储服务Cinder(转)

http://www.sxt.cn/u/4678/blog/5483   0。验证环境 环境包括: 1、一个controller节点,运行nova-api, nova-scheduler,...
  • lic95
  • lic95
  • 2015年10月22日 14:45
  • 1306

[管理篇]VMWare搭建Openstack——Cinder完成对云硬盘的扩容

前面提到了,如果创建更多的时候,或者你的大小过大(X GB),系统也会提示失败,失败信息包括:Stderr: u'  Volume group "cinder-volumes" has insuffi...

openstack常见问题解决办法

作者:【吴业亮】云计算开发工程师 博客:http://blog.csdn.net/wylfengyujiancheng 平时小伙伴们经常问我关于openstack问题,感谢小伙伴们的信任,现总结如...

service openstack-cinder-volume status

openstack-cinder-volume dead but pid file exists 问题:service openstack-cinder-volume status     ope...

[openstack]创建卷(create volume)流程

图1 create volu

OpenStack块存储nova-volume工作机制和相关问题

最近有点忙,收到不少邮件或者留言,有些没有回复,在此致歉。在OpenStack中,出现问题比较多的除了网络部分,还有存储部分。对于swift对象存储的研究和相关技术资料已经有很多,而块存储nova-v...
  • hilyoo
  • hilyoo
  • 2012年07月25日 12:54
  • 29349

OpenStack Grizzly实例重启之后cinder-volume服务无法启动的解决办法

问题描述:Cinder组件装在Controller节点,节点发生故障重启后,发现挂载了volumn的实例无法启动,并且无法创建及挂载定的volume,查看cinder-volume服务,处于stop状...
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:[问题篇]VMWare搭建Openstack——Cinder创建扩展Volume的状态Error的问题
举报原因:
原因补充:

(最多只允许输入30个字)