本文是在另外一篇文章的基础上使用kolla-ansible 部署octavia实践总结。部分内容总结略有粗糙,见谅。
使用kolla-ansible all-in-one部署OpenStack Stein
https://blog.csdn.net/zongzw/article/details/106952948
参考链接
https://www.lijiawang.org/posts/kolla-octavia.html
kolla-octavia
(5条消息)openstack kolla 方式配置octavia_运维_weixin_40161962的博客-CSDN博客
OpenStack Docs: Octavia Documentation
OpenStack Docs: Octavia Certificate Configuration Guide
OpenStack Docs: Using Octavia CLI extensions to OpenStack Client
OpenStack Docs: Python Octavia Client Reference
部署过程
下载octavia 源码
下载octavia 代码,主要会用到其中的生成cert部分,但需要 注意的是
部署stein版本需要加 -b stable/stein
# git clone https://github.com/openstack/octavia -b stable/stein
生成octavia需要的key cert集合
- 使用/etc/kolla/passwords.yml中的octavia_ca_password替换bin/create_certificates.sh脚本中的foobar
# git clone https://review.openstack.org/p/openstack/octavia
# cd octavia
# grep octavia_ca /etc/kolla/passwords.yml
octavia_ca_password: mEUyBHLopKk501CX30WRnPuiDmoP3I7eNQIQbC6z
# sed -i 's/foobar/mEUyBHLopKk501CX30WRnPuiDmoP3I7eNQIQbC6z/g' bin/create_certificates.sh
# ./bin/create_certificates.sh cert $(pwd)/etc/certificates/openssl.cnf
- 生成的证书需要 重命名
命令如下:
bin/create_certificates.sh /etc/kolla/config/octavia `pwd`/etc/certificates/openssl.cnf
.
|-- ca_01.pem
|-- cakey.pem <- from private cakey.pem
|-- client.csr
|-- client.pem
|-- index.txt
|-- index.txt.attr
|-- index.txt.old
|-- newcerts
| `-- 01.pem
|-- private
|-- serial
|-- serial.old
|-- server.key <- from client.key
`-- server.pem <- from client-.pem
修改/etc/kolla/globals.yml
将这个选项改成yes: enable_octavia
依次重新运行部署命令
# (kolla) kolla-ansible bootstrap-servers
# (kolla) kolla-ansible prechecks
# (kolla) kolla-ansible deploy
# (kolla) kolla-ansible post-deploy
openstack client 中 添加loadbalancer 子命令集合
此命令在openstack virtualenv中执行
# pip install python-octaviaclient
创建amphora 镜像
创建 amphora 的过程是在部署完成后执行的。
和https://github.com/openstack/octavia/tree/master/diskimage-create
制作image
执行安装依赖包命令,此命令中的依赖包会在pip install -r requirements.txt 中需要。
yum -y install python-devel libffi-devel gcc openssl-devel libselinux-python
# virtualenv /root/venv/octavia
# source venv/octavia/bin/activate
(octavia) [root@kolla]
# pip install -U pip
# cd octavia/
# pip install win-inet-pton # 不执行此步会有安装错误报出
# pip install -r requirements.txt
# cd diskimage-create/
# pip install -r requirements.txt
# fix the error:
# 2020-06-24 02:31:51.830 | qcow2 output format specified but qemu-img executable not found.
# yum install -y qemu-img
# fix the error:
# line 39: debootstrap: command not found
# yum install debootstrap
# ./diskimage-create.sh -i ubuntu -t qcow2 -o amphora-x64-haproxy
不明白为什么这个过程不能自动化到kolla-ansible的安装部署过程中。上边这个命令也是从网上过来人的blog中凑来的。。。。
openstack image create --container-format bare --disk-format qcow2 --private --file /root/octavia/diskimage-create/amphora-x64-haproxy.qcow2 --tag amphora amphora
创建octavia_ssh_key
openstack keypair create --public-key /root/.ssh/id_rsa.pub octavia_ssh_key
修改octavia.conf文件
octavia四个组件的目录下都有这个文件,我们需要修改的是octavia-worker下的octavia.conf
amp_boot_network_list = 网络ID
amp_secgroup_list = 安全组ID
amp_flavor_id = 实例FlavorID
docker restart octavia_worker
创建loabalancer尝试
openstack loadbalancer create --vip-network-id 8381dc8d-2122-4379-821f-b706564e580e
等待许久后会失败,因为网络问题amphora 虚机中无法连接到controller 节点 10.145.64.104:5555,参考问题与解决部分。
使用horizon操作octavia
似曾相识,跟neutron lbaas 一致:
Octavia 可以从horizon和 CLI两种方式操作,创建LB后,在Compute -> Instance中可以看到有amphora的instance启动,其中集成了haproxy。
Octavia将loadbalancer的能力交给tenant,用户可以自己创建LB,其实现本质是在compute上创建一个带有负载均衡功能的虚机(默认负载均衡器为haproxy)VIP为floatingIP,并通过此虚机实现负载均衡能力。
问题与解决
问题:执行bootstrap_servers时异常。
以下错误输出问题很多,省略部分输出。
TASK [octavia : include_tasks] ************************************************************************************
included: /root/venv/kolla/share/kolla-ansible/ansible/roles/octavia/tasks/bootstrap_service.yml for localhost
TASK [octavia : Running Octavia bootstrap container] **************************************************************
fatal: [localhost -> localhost]: FAILED! => {"changed": true, "msg": "Container exited with non-zero return code 1", "rc": 1, "stderr": "+ sudo -E kolla_set_configs\nINFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json\nINFO:__main__:Validating config file\nINFO:__main__:Kolla ....
.....
raise_mysql_exception\n2020-04-27 05:42:38.221 14 ERROR octavia-db-manage raise errorclass(errno, errval)\n2020-04-27 05:42:38.221 14 ERROR octavia-db-manage OperationalError: (pymysql.err.OperationalError) (1045, u\"Access denied for user 'octavia'@'kolla.pdsea.f5net.com' (using password: YES)\") (Background on this error at: http://sqlalche.me/e/e3q8)\n2020-04-27 05:42:38.221 14 ERROR octavia-db-manage \u001b[00m\n", "stdout_lines": ["2020-04-27 05:42:38.221 14 CRITICAL octavia-db-manage [-] Unhandled error: OperationalError:
....
"2020-04-27 06:39:29.446 13 ERROR octavia-db-manage OperationalError: (pymysql.err.OperationalError) (1045, u\"Access denied for user 'octavia'@'kolla.pdsea.f5net.com' (using password: YES)\") (Background on this error at: http://sqlalche.me/e/e3q8)",
"2020-04-27 06:39:29.446 13 ERROR octavia-db-manage \u001b[00m"
]
}
解决:
手动pull docker image: kolla-ansible pull.
奇怪的是我登录dockerhub,没有发现这四个docker image。
[root@kolla ~]# docker images| grep octavia
kolla/centos-source-octavia-api stein 838b0afb0fa0 18 hours ago 897MB
kolla/centos-source-octavia-housekeeping stein 9c82e759f2c6 18 hours ago 835MB
kolla/centos-source-octavia-worker stein e9b57845d6b5 18 hours ago 835MB
kolla/centos-source-octavia-health-manager stein d0c49699d0ff 18 hours ago 835MB
之上的错误很可能 是我执行的是kolla-ansible upgrade,而不是kolla-ansible deploy
问题:debootstrap: command not found
生成amphora image的时候 diskimage_create.sh命令执行失败。
2020-04-28 02:38:12.477 | /tmp/dib_build.VhHboWib/hooks/root.d/08-debootstrap: line 39: debootstrap: command not found
yum install debootstrap
问题: 创建lb失败,因为amphora虚机连接不到controller
openstack image create --container-format bare --disk-format qcow2 --private --file /root/octavia/diskimage-create/amphora-x64-haproxy.qcow2 --tag amphora amphora
openstack loadbalancer create --vip-network-id 8381dc8d-2122-4379-821f-b706564e580e
等待许久后会失败,因为网络问题amphora 虚机中无法连接到controller 节点 10.145.64.104:5555
问题:Could not find or access '/etc/kolla/config/octavia/cakey.pem'
TASK [octavia : Copying certificate files for octavia-worker] ******************************************************
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: If you are using a module and expect the file to exist on the remote, see the remote_src option
failed: [localhost] (item=cakey.pem) => {"ansible_loop_var": "item", "changed": false, "item": "cakey.pem", "msg": "Could not find or access '/etc/kolla/config/octavia/cakey.pem' on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: If you are using a module and expect the file to exist on the remote, see the remote_src option
failed: [localhost] (item=ca_01.pem) => {"ansible_loop_var": "item", "changed": false, "item": "ca_01.pem", "msg": "Could not find or access '/etc/kolla/config/octavia/ca_01.pem' on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: If you are using a module and expect the file to exist on the remote, see the remote_src option
failed: [localhost] (item=client.pem) => {"ansible_loop_var": "item", "changed": false, "item": "client.pem", "msg": "Could not find or access '/etc/kolla/config/octavia/client.pem' on the Ansible Controller.\nIf you are using a module and expect the file to exist on the remote, see the remote_src option"}
问题:The request you have made requires authentication. (HTTP 401)
无法完成loadbalancer的部署
The request you have made requires authentication. (HTTP 401) (Request-ID: req-552f5d1b-1f85-42a8-a60f-a922fb5fcc19) (HTTP 500) (Request-ID: req-3ed62325-153f-4857-8ce6-7f8b1063c6df)
需要添加权限:
openstack role add --project admin --user octavia admin
将octavia用户添加到admin 项目中,从这里也可以看出 创建负载均衡器(虚机 网络)操作是用的octavia?
问题:horizon 出现错误,部分页面无法显示
在firefox或者chrome 的页面调试中可以看到
Uncaught Error: [$injector:modulerr] Failed to instantiate module horizon.app due to:
Error: [$injector:nomod] Module 'horizon.app' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument.
。。。。。
horizon的代码位置 进入到horizon container中,查看/etc/httpd/conf.d/horizon.conf
但是实在没有时间去搞这些个破问题,简单粗暴一点搞定:
# docker rm --force horizon
# docker rmi kolla/centos-source-horizon:stein
然后重新:
kolla-ansible bootstrap-servers
kolla-ansible prechecks
kolla-ansible deploy
kolla-ansible post-deploy
稍等horizon运行3-4分钟后,重新访问UI,OK了。