一、搭建高可用OpenStack（Queen版）集群之部署Nova控制节点集群
一、简介
1、概述
使用Openstack计算服务来托管和管理云计算系统。Openstack计算服务是基础设施即服务(IaaS)系统的主要部分，模块主要由python实现。
Openstack计算组件请求Openstack Identitiy服务进行认证；请求Openstack Image服务提供磁盘镜像；为Openstack dashboard提供用户和管理员接口。磁盘镜像访问限制在项目与用户上；配额以每个项目进行设定。Openstack组件可以在标准硬件上水平大规模扩展，并且下载磁盘镜像启动虚拟机实例。
2、Openstack计算服务由下列组件所构成
（1） nova-api 服务：
接收和响应来自最终用户的计算API请求。此服务支持Openstack计算服务API，例如启动一个实例。
（2）nova-api-metadata 服务
接受来自虚拟机发送的元数据请求。
（3）nova-compute 服务
一个持续工作的守护进程，通过Hypervior的API来创建和销毁虚拟机实例，例如：
XenServer 的 XenAPI
KVM或QEMU的libvirt
过程是蛮复杂的。最为基本的，守护进程同意了来自队列的动作请求，转换为一系列的系统命令如启动一个KVM实例，然后到数据库中更新它的状态。
（4）nova-scheduler 服务
拿到一个来自队列请求虚拟机实例，然后决定那台物理计算节点来运行它。简单的说，该服务通过算法来决定在哪个计算节点来启动虚拟机。
（5）nova-cert 模块
Nova-network worker守护进程
从队列中接受网络任务，并操作网络。执行任务。例如创建网桥的接口或者改变iptables的规则。该功能应该被网络服务neutron取代。
Nova-consoleauth 守护进程
授权控制台代理所提供的用户令牌。
Nova-novncproxy 守护进程
提供一个代理，用于访问正在运行的实例，通过VNC协议，支持基于浏览器的novnc客户端。
队列
一个守护进程间传递消息的组件。前面部署的rabbitmq-server
SQL数据库
存储构建时和运行时的状态，为云基础设施，包括有：
可用实例类型
使用中的实例
可用网络
项目
理论上，OpenStack计算可以支持任何和SQL-Alchemy所支持的后端数据库，通常使用SQLite3来做测试可开发工作，MySQL和PostgreSQL 作生产环境。
二、部署Nova控制节点集群
1、创建nova相关数据库
在任意控制节点创建数据库，后台数据自动同步
nova服务含4个数据库，统一授权到nova用户；
placement主要涉及资源统筹，较常用的api接口是获取备选资源与claim资源等
mysql -u root -p
CREATE DATABASE nova_api; CREATE DATABASE nova; CREATE DATABASE nova_cell0; CREATE DATABASE nova_placement; GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'localhost' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON nova_api.* TO 'nova'@'%' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'localhost' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON nova_cell0.* TO 'nova'@'%' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON nova_placement.* TO 'nova'@'localhost' IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON nova_placement.* TO 'nova'@'%' IDENTIFIED BY '123456'; flush privileges; exit;
2、创建nova/placement-api
在任意控制节点操作
调用nova相关服务需要认证信息，加载环境变量脚本即可
. admin-openrc
1、创建nova/plcement用户
service项目已在glance章节创建；
nova/placement用户在”default” domain中
[root@controller01 ~]# openstack user create --domain default --password=nova_pass nova
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | 2edf2fc7f9404e03bb33495957987617 |
| name | nova |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
[root@controller01 ~]# openstack user create --domain default --password=placement_pass placement
+---------------------+----------------------------------+
| Field | Value |
+---------------------+----------------------------------+
| domain_id | default |
| enabled | True |
| id | ab59d08c20c54bc192f55682158a0fa1 |
| name | placement |
| options | {} |
| password_expires_at | None |
+---------------------+----------------------------------+
2、nova/placement赋权
为nova/placement用户赋予admin权限（没有返回值）
openstack role add --project service --user nova admin openstack role add --project service --user placement admin
3、创建nova/placement服务实体
nova服务实体类型”compute”；
placement服务实体类型”placement”
[root@controller01 ~]# openstack service create --name nova --description "OpenStack Compute" compute +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | OpenStack Compute | | enabled | True | | id | 2160b3d55a8c42deb0e53d88f9a60893 | | name | nova | | type | compute | +-------------+----------------------------------+ [root@controller01 ~]# openstack service create --name placement --description "Placement API" placement +-------------+----------------------------------+ | Field | Value | +-------------+----------------------------------+ | description | Placement API | | enabled | True | | id | 27001c0b666440968996d9c832dee394 | | name | placement | | type | placement | +-------------+----------------------------------+
4、创建nova/placement-api
注意
- --region与初始化admin用户时生成的region一致；
- api地址统一采用vip，如果public/internal/admin分别使用不同的vip，请注意区分；
- nova-api 服务类型为compute，placement-api服务类型为placement；
# nova public api openstack endpoint create --region RegionTest compute public http://controller:8774/v2.1 # nova internal api openstack endpoint create --region RegionTest compute internal http://controller:8774/v2.1 # nova admin api openstack endpoint create --region RegionTest compute admin http://controller:8774/v2.1 # placement public api openstack endpoint create --region RegionTest placement public http://controller:8778 # placement internal api openstack endpoint create --region RegionTest placement internal http://controller:8778 # placement admin api openstack endpoint create --region RegionTest placement admin http://controller:8778
3、安装nova
在全部控制节点安装nova相关服务
yum install openstack-nova-api openstack-nova-conductor \ openstack-nova-console openstack-nova-novncproxy \ openstack-nova-scheduler openstack-nova-placement-api -y
4、配置nova.conf
在全部控制节点操作
注意：
- ”my_ip”参数，根据节点修改；
- nova.conf文件的权限：root:nova
cp -rp /etc/nova/nova.conf{,.bak}
egrep -v "^$|^#" /etc/nova/nova.conf
[DEFAULT] my_ip=10.20.9.189 use_neutron=true firewall_driver=nova.virt.firewall.NoopFirewallDriver enabled_apis=osapi_compute,metadata osapi_compute_listen=$my_ip osapi_compute_listen_port=8774 metadata_listen=$my_ip metadata_listen_port=8775 # 前端采用haproxy时，服务连接rabbitmq会出现连接超时重连的情况，可通过各服务与rabbitmq的日志查看； # transport_url=rabbit://openstack:openstack@controller:5673 # rabbitmq本身具备集群机制，官方文档建议直接连接rabbitmq集群；但采用此方式时服务启动有时会报错，原因不明；如果没有此现象，强烈建议连接rabbitmq直接对接集群而非通过前端haproxy transport_url=rabbit://openstack:openstack@controller01:5672,openstack:openstack@controller02:5672,openstack:openstack@controller03:5672 [api] auth_strategy=keystone [api_database] connection=mysql+pymysql://nova:123456@controller/nova_api [barbican] [cache] backend=oslo_cache.memcache_pool enabled=True memcache_servers=controller01:11211,controller02:11211,controller03:11211 [cells] [cinder] [compute] [conductor] [console] [consoleauth] [cors] [crypto] [database] connection = mysql+pymysql://nova:123456@controller/nova [devices] [ephemeral_storage_encryption] [filter_scheduler] [glance] api_servers = http://controller:9292 [guestfs] [healthcheck] [hyperv] [ironic] [key_manager] [keystone] [keystone_authtoken] auth_uri = http://controller:5000 auth_url = http://controller:35357 memcached_servers = controller01:11211,controller02:11211,controller03:11211 auth_type = password project_domain_name = default user_domain_name = default project_name = service username = nova password = nova_pass [libvirt] [matchmaker_redis] [metrics] [mks] [neutron] [notifications] [osapi_v21] [oslo_concurrency] lock_path=/var/lib/nova/tmp [oslo_messaging_amqp] [oslo_messaging_kafka] [oslo_messaging_notifications] [oslo_messaging_rabbit] [oslo_messaging_zmq] [oslo_middleware] [oslo_policy] [pci] [placement] region_name = RegionTest project_domain_name = Default project_name = service auth_type = password user_domain_name = Default auth_url = http://controller:35357/v3 username = placement password = placement_pass [quota] [rdp] [remote_debug] [scheduler] [serial_console] [service_user] [spice] [upgrade_levels] [vault] [vendordata_dynamic_auth] [vmware] [vnc] enabled=true server_listen=$my_ip server_proxyclient_address=$my_ip novncproxy_base_url=http://$my_ip:6080/vnc_auto.html novncproxy_host=$my_ip novncproxy_port=6080 [workarounds] [wsgi] [xenserver] [xvp]
同步配置完毕，更改IP
5、配置00-nova-placement-api.conf
在全部控制节点操作
注意根据不同节点修改监听地址
sed -i "s/Listen\ 8778/Listen\ 10.20.9.189:8778/g" /etc/httpd/conf.d/00-nova-placement-api.conf sed -i "s/*:8778/10.20.9.189:8778/g" /etc/httpd/conf.d/00-nova-placement-api.conf
cp -rp /etc/httpd/conf.d/00-nova-placement-api.conf{,.bak}
echo "
#Placement API
<Directory /usr/bin>
<IfVersion >= 2.4>
Require all granted
</IfVersion>
<IfVersion < 2.4>
Order allow,deny
Allow from all
</IfVersion>
</Directory>
" >> /etc/httpd/conf.d/00-nova-placement-api.conf
同步配置完毕，更改IP
#02 sed -i "s/10.20.9.189/10.20.9.190/g" /etc/httpd/conf.d/00-nova-placement-api.conf #03 sed -i "s/10.20.9.189/10.20.9.45/g" /etc/httpd/conf.d/00-nova-placement-api.conf
重启httpd服务，启动placement-api监听端口
systemctl restart httpd
6、同步nova相关数据库
1、同步nova相关数据库
# 同步nova-api数据库 su -s /bin/sh -c "nova-manage api_db sync" nova # 注册cell0数据库 su -s /bin/sh -c "nova-manage cell_v2 map_cell0" nova # 创建cell1 cell (返回55c43661-28f2-4efa-a3d0-12d4b2769fa3)
su -s /bin/sh -c "nova-manage cell_v2 create_cell --name=cell1 --verbose" nova # 同步nova数据库； # 忽略”deprecated”信息 su -s /bin/sh -c "nova-manage db sync" nova
补充：此版本在向数据库同步导入数据表时，报错：
/usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:332: NotSupportedWarning: Configuration option(s) ['use_tpool'] not supported exception.NotSupportedWarning
解决参考链接：腾讯云 产业智变·云启未来 - 腾讯
根据报错信息找到对应的文件，注释第325、329、330、331、332和333行的配置。
sed -n '325,333p' /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py
结果如下
# if not_supported: # # would like to raise ValueError here, but there are just # # too many unrecognized (obsolete?) configuration options # # coming in from projects # warnings.warn( # "Configuration option(s) %r not supported" % # sorted(not_supported), # exception.NotSupportedWarning # )
7、启动服务
在全部控制节点操作
开机自启
systemctl enable openstack-nova-api.service \ openstack-nova-consoleauth.service \ openstack-nova-scheduler.service \ openstack-nova-conductor.service \ openstack-nova-novncproxy.service
启动
systemctl restart openstack-nova-api.service systemctl restart openstack-nova-consoleauth.service systemctl restart openstack-nova-scheduler.service systemctl restart openstack-nova-conductor.service systemctl restart openstack-nova-novncproxy.service
查看状态
systemctl status openstack-nova-api.service \ openstack-nova-consoleauth.service \ openstack-nova-scheduler.service \ openstack-nova-conductor.service \ openstack-nova-novncproxy.service
查看端口
netstat -tunlp | egrep '8774|8775|8778|6080'
8、验证
在任意控制节点操作
. admin-openrc
列出各服务组件，查看状态
使用命令nova service-list 或者openstack compute service list
[root@controller01 ~]# openstack compute service list
+-----+------------------+--------------+----------+---------+-------+----------------------------+
| ID | Binary | Host | Zone | Status | State | Updated At |
+-----+------------------+--------------+----------+---------+-------+----------------------------+
| 85 | nova-consoleauth | controller01 | internal | enabled | up | 2018-10-15T09:41:58.000000 |
| 88 | nova-consoleauth | controller03 | internal | enabled | up | 2018-10-15T09:42:00.000000 |
| 91 | nova-consoleauth | controller02 | internal | enabled | up | 2018-10-15T09:42:00.000000 |
| 94 | nova-scheduler | controller01 | internal | enabled | up | 2018-10-15T09:42:02.000000 |
| 97 | nova-scheduler | controller03 | internal | enabled | up | 2018-10-15T09:42:04.000000 |
| 100 | nova-scheduler | controller02 | internal | enabled | up | 2018-10-15T09:42:04.000000 |
| 103 | nova-conductor | controller01 | internal | enabled | up | 2018-10-15T09:42:05.000000 |
| 115 | nova-conductor | controller03 | internal | enabled | up | 2018-10-15T09:41:58.000000 |
| 124 | nova-conductor | controller02 | internal | enabled | up | 2018-10-15T09:41:58.000000 |
+-----+------------------+--------------+----------+---------+-------+----------------------------+
展示api端点
[root@controller01 ~]# openstack catalog list +-----------+-----------+-----------------------------------------+ | Name | Type | Endpoints | +-----------+-----------+-----------------------------------------+ | nova | compute | RegionTest | | | | public: http://controller:8774/v2.1 | | | | RegionTest | | | | internal: http://controller:8774/v2.1 | | | | RegionTest | | | | admin: http://controller:8774/v2.1 | | | | | | keystone | identity | RegionTest | | | | internal: http://controller:5000/v3/ | | | | RegionTest | | | | public: http://controller:5000/v3/ | | | | RegionTest | | | | admin: http://controller:35357/v3/ | | | | | | glance | image | RegionTest | | | | internal: http://controller:9292 | | | | RegionTest | | | | admin: http://controller:9292 | | | | RegionTest | | | | public: http://controller:9292 | | | | | | placement | placement | RegionTest | | | | public: http://controller:8778 | | | | RegionTest | | | | admin: http://controller:8778 | | | | RegionTest | | | | internal: http://controller:8778 | | | | | +-----------+-----------+-----------------------------------------+
检查cell与placement api运行正常
[root@controller01 ~]# nova-status upgrade check +--------------------------------+ | Upgrade Check Results | +--------------------------------+ | Check: Cells v2 | | Result: Success | | Details: None | +--------------------------------+ | Check: Placement API | | Result: Success | | Details: None | +--------------------------------+ | Check: Resource Providers | | Result: Success | | Details: None | +--------------------------------+ | Check: Ironic Flavor Migration | | Result: Success | | Details: None | +--------------------------------+ | Check: API Service Version | | Result: Success | | Details: None | +--------------------------------+
9、设置pcs资源
在任意控制节点操作
1、添加资源openstack-nova-api，openstack-nova-consoleauth，openstack-nova-scheduler，openstack-nova-conductor与openstack-nova-novncproxy
pcs resource create openstack-nova-api systemd:openstack-nova-api --clone interleave=true pcs resource create openstack-nova-consoleauth systemd:openstack-nova-consoleauth --clone interleave=true pcs resource create openstack-nova-scheduler systemd:openstack-nova-scheduler --clone interleave=true pcs resource create openstack-nova-conductor systemd:openstack-nova-conductor --clone interleave=true pcs resource create openstack-nova-novncproxy systemd:openstack-nova-novncproxy --clone interleave=true
经验证，建议openstack-nova-api，openstack-nova-consoleauth，openstack-nova-conductor与openstack-nova-novncproxy 等无状态服务以active/active模式运行；openstack-nova-scheduler等服务以active/passive模式运行
2、查看pcs资源
[root@controller01 ~]# pcs resource
vip (ocf::heartbeat:IPaddr2): Started controller01
Clone Set: lb-haproxy-clone [lb-haproxy]
Started: [ controller01 ]
Stopped: [ controller02 controller03 ]
Clone Set: openstack-keystone-clone [openstack-keystone]
Started: [ controller01 controller02 controller03 ]
Clone Set: openstack-glance-api-clone [openstack-glance-api]
Started: [ controller01 controller02 controller03 ]
Clone Set: openstack-glance-registry-clone [openstack-glance-registry]
Started: [ controller01 controller02 controller03 ]
Clone Set: openstack-nova-api-clone [openstack-nova-api]
Started: [ controller01 controller02 controller03 ]
Clone Set: openstack-nova-consoleauth-clone [openstack-nova-consoleauth]
Started: [ controller01 controller02 controller03 ]
Clone Set: openstack-nova-scheduler-clone [openstack-nova-scheduler]
Started: [ controller01 controller02 controller03 ]
Clone Set: openstack-nova-conductor-clone [openstack-nova-conductor]
Started: [ controller01 controller02 controller03 ]
Clone Set: openstack-nova-novncproxy-clone [openstack-nova-novncproxy]
Started: [ controller01 controller02 controller03 ]
1634
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
