DevStack 部署 OpenStack 多节点

willops

已于 2024-03-24 14:29:12 修改

阅读量1.2k

点赞数 28

分类专栏： OpenStack 文章标签： openstack

于 2024-03-24 13:44:46 首次发布

本文链接：https://blog.csdn.net/networken/article/details/136974688

版权

OpenStack 专栏收录该内容

19 篇文章 17 订阅

订阅专栏

DevStack 部署 OpenStack 多节点

DevStack 支持OpenStack多节点部署，下面以一个控制节点和一个计算节点为例，介绍多节点多网卡部署流程。

官方文档：

https://docs.openstack.org/devstack/latest/guides/multinode-lab.html

https://docs.openstack.org/neutron/latest/contributor/testing/ml2_ovn_devstack.html

节点规划

基于Ubuntu 22.04 操作系统，使用 Devstack 部署工具搭建多节点的 OpenStack 开发环境。

节点规划，devstack21 为控制节点，devstack22为计算节点。

主机名	节点IP	CPU	内存	磁盘	网卡	OS	角色
devstack21	192.168.93.21	4C	8G	100G	2	Ubuntu22.04	控制节点
devstack22	192.168.93.22	4C	8G	100G	2	Ubuntu22.04	计算节点

网段规划，规划两个网络平面，对应管理网络和外部网络。租户网络（geneve隧道网络）与管理网络合并。

网段	网关	网卡模式	网络类型
192.168.93.0/24	192.168.93.2	NAT	管理网络
192.168.72.0/24	192.168.72.8	桥接	外部网络

网卡规划，每个节点两块网卡，分别对应管理网络和外部网络。

主机名	网卡	IP地址	网卡模式	网络类型
devstack01	ens33	192.168.93.21/24	NAT	管理网络
	ens37	NA	桥接	外部网络
devstack02	ens33	192.168.93.22/24	NAT	管理网络
	ens37	NA	桥接	外部网络

网络架构图如下：
在这里插入图片描述
说明：

管理网络需要能够访问外网，部署时需要通过该网络下载依赖，所以选择NAT模式
外部网络也可以选择hostonly模式，外部网络不一定是公网

测试环境： VMware Workstation 17
在这里插入图片描述
说明：

准备两块网卡，注意配置网卡模式
也可以勾选处理器–>虚拟化引擎选项，提高openstack创建的实例性能

前置准备

以下操作在所有节点执行。

配置http代理（可选，需要科技）

cat >/etc/profile.d/proxy.sh<<EOF
export http_proxy="http://192.168.93.1:7890"
export https_proxy="http://192.168.93.1:7890"
export no_proxy="192.168.93.0/24,10.0.0.0/8,192.168.72.0/24,localhost,127.0.0.0/8,.localdomain"
EOF
source /etc/profile

说明：否则你可能需要自行配置国内的apt源、pip源，例如阿里云国内源，切换git仓库源码地址为trystack源，另外devstack直接下载二进制文件时你可能只能使用export http_proxy=xxx这种方式，例如stack.sh脚本会执行以下操作。

wget --progress=dot:giga -t 2 -c https://github.com/etcd-io/etcd/releases/download/v3.4.27/etcd-v3.4.27-linux-amd64.tar.gz -O /opt/stack/devstack/files/etcd-v3.4.27-linux-amd64.tar.gz

系统更新

apt update -y && apt upgrade -y

配置主机名

hostnamectl set-hostname devstack21
hostnamectl set-hostname devstack22

配置时间同步

apt install -y chrony
timedatectl set-timezone Asia/Shanghai

devstack21节点网卡配置示例:

root@devstack21:~# cat /etc/netplan/00-installer-config.yaml
network:
  ethernets:
    ens33:
      addresses:
      - 192.168.93.21/24
      nameservers:
        addresses:
        - 223.5.5.5
        - 223.6.6.6
        search: []
      routes:
        - to: default
          via: 192.168.93.2
    ens37:
      dhcp4: false
  version: 2

devstack21网卡信息示例如下:

root@devstack21:~# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:b0:da:db brd ff:ff:ff:ff:ff:ff
    altname enp2s1
    inet 192.168.93.21/24 brd 192.168.93.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:feb0:dadb/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:b0:da:e5 brd ff:ff:ff:ff:ff:ff
    altname enp2s5
    inet6 fe80::20c:29ff:feb0:dae5/64 scope link 
       valid_lft forever preferred_lft forever

说明：devstack22节点配置类似，仅IP不同。

添加 Stack 用户

以下操作在所有节点执行。

sudo useradd -s /bin/bash -d /opt/stack -m stack

确保 stack 用户的主目录对所有人都具有可执行权限，因为基于 RHEL 的发行版使用 700 创建它，而 Ubuntu 21.04+ 使用 750 创建它，这可能会在运行期间导致问题部署。

sudo chmod +x /opt/stack

由于该用户将对您的系统进行许多更改，因此它应该具有 sudo 权限：

echo "stack ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/stack
sudo -u stack -i

此时会切换到stack用户视图下。

下载 DevStack

以下操作在所有节点执行。

下载devstack源码

stack@devstack21:~$ pwd
/opt/stack
stack@devstack21:~$ git clone https://opendev.org/openstack/devstack

下载neutron源码

stack@devstack21:~$ pwd
/opt/stack
stack@devstack21:~$ git clone https://opendev.org/openstack/neutron.git

安装控制节点

以下操作在控制节点执行。

配置 DevStack 以使用 OVN 驱动程序。OVN 驱动程序附带了一个示例 DevStack 配置文件，位于neutron项目代码下。

复制neutron示例文件到devstack目录下：

stack@devstack21:~$ cd devstack
stack@devstack21:~/devstack$ pwd
/opt/stack/devstack
stack@devstack21:~$ egrep -v "^#|^$" ../neutron/devstack/ovn-local.conf.sample > ./local.conf

配置文件修改后如下

cat >local.conf<<'EOF'
[[local|localrc]]
# git_base
GIT_BASE="https://github.com"
#GIT_BASE=https://opendev.org
#GIT_BASE=http://git.trystack.cn

# password
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=$ADMIN_PASSWORD

HOST_IP=192.168.93.21
SERVICE_HOST=192.168.93.21
MYSQL_HOST=$SERVICE_HOST
RABBIT_HOST=$SERVICE_HOST
GLANCE_HOSTPORT=$SERVICE_HOST:9292

Q_AGENT=ovn
Q_ML2_PLUGIN_MECHANISM_DRIVERS=ovn,logger
Q_ML2_PLUGIN_TYPE_DRIVERS=local,flat,vlan,geneve
Q_ML2_TENANT_NETWORK_TYPE="geneve"
LOGFILE=$DEST/logs/stack.sh.log
enable_service ovn-northd
enable_service ovn-controller
enable_service q-ovn-metadata-agent
enable_service q-svc
disable_service q-agt
disable_service q-l3
disable_service q-dhcp
disable_service q-meta
enable_plugin neutron https://github.com/openstack/neutron
enable_service q-trunk
enable_service q-dns
enable_service q-port-forwarding
enable_service q-qos
enable_service neutron-segments
enable_service q-log
enable_plugin neutron-tempest-plugin https://github.com/openstack/neutron-tempest-plugin
enable_service horizon
disable_service cinder c-sch c-api c-vol
OVN_BUILD_MODULES=True
ENABLE_CHASSIS_AS_GW=True

## Neutron options
FLOATING_RANGE="192.168.72.0/24"
Q_FLOATING_ALLOCATION_POOL=start=192.168.72.110,end=192.168.72.120
PUBLIC_NETWORK_GATEWAY="192.168.72.8"
PUBLIC_INTERFACE=ens37

# Open vSwitch provider networking configuration
Q_USE_PROVIDERNET_FOR_PUBLIC=True
OVS_PHYSICAL_BRIDGE=br-ex
PUBLIC_BRIDGE=br-ex
OVS_BRIDGE_MAPPINGS=public:br-ex

[[post-config|$NOVA_CONF]]
[scheduler]
discover_hosts_in_cells_interval = 2
EOF

主要配置参数说明：

GIT_BASE: 网络不好时可尝试使用GIT_BASE=http://git.trystack.cn
HOST_IP: 本机IP地址
FLOATING_RANGE：用于 GUEST VM 访问外部网络，以通过向GUEST VM分配 FLOATING IP 来从外部网络访问GUEST VM。
Q_FLOATING_ALLOCATION_POOL：为devstack分配指定的外部IP地址范围，因为本地其他机器也在使用该物理网段。
PUBLIC_INTERFACE: 外部网络网卡名称
PUBLIC_NETWORK_GATEWAY：外部网络网关地址
Q_USE_PROVIDERNET_FOR_PUBLIC：允许配置Neutron L3 Agent的外部接口使用provider网络，从而实现对OpenStack云实例的公共网络访问。
OVS_PHYSICAL_BRIDGE：指定了一个OVS网桥br-ex作为物理网桥，外部网络的流量将通过这个OVS网桥br-ex，底层物理网卡将连接到br-ex网桥上
PUBLIC_BRIDGE：直接将之前定义的br-ex网桥指定为公共网络的网桥，所有外部网络的流量都将通过br-ex网桥
OVS_BRIDGE_MAPPINGS：映射外部网络到之前定义的br-ex网桥

安装控制节点

$ ./stack.sh

确认安装成功

......
=========================
DevStack Component Timing
 (times are in seconds)  
=========================
wait_for_service      12
async_wait           102
osc                  180
apt-get               71
test_with_retry        5
dbsync                 7
pip_install          128
apt-get-update         1
run_process           27
git_timed             85
-------------------------
Unaccounted time     184
=========================
Total runtime        802

=================
 Async summary
=================
 Time spent in the background minus waits: 322 sec
 Elapsed time: 802 sec
 Time if we did everything serially: 1124 sec
 Speedup:  1.4015


Post-stack database query stats:
+------------+-----------+-------+
| db         | op        | count |
+------------+-----------+-------+
| keystone   | SELECT    | 31269 |
| keystone   | INSERT    |    82 |
| neutron    | SELECT    |  4482 |
| neutron    | CREATE    |     1 |
| neutron    | SHOW      |     4 |
| neutron    | INSERT    |  4113 |
| neutron    | DELETE    |    27 |
| neutron    | UPDATE    |   116 |
| placement  | SELECT    |    38 |
| placement  | INSERT    |    55 |
| placement  | SET       |     1 |
| nova_api   | SELECT    |   305 |
| nova_cell0 | SELECT    |    65 |
| nova_cell1 | SELECT    |   317 |
| nova_cell0 | INSERT    |     5 |
| nova_cell0 | UPDATE    |     3 |
| nova_cell1 | INSERT    |     4 |
| nova_cell1 | UPDATE    |    20 |
| placement  | UPDATE    |     3 |
| glance     | SELECT    |    57 |
| glance     | INSERT    |    14 |
| glance     | UPDATE    |     2 |
| nova_api   | INSERT    |    23 |
| nova_api   | SAVEPOINT |    11 |
| nova_api   | RELEASE   |    11 |
+------------+-----------+-------+



This is your host IP address: 192.168.93.21
This is your host IPv6 address: ::1
Horizon is now available at http://192.168.93.21/dashboard
Keystone is serving at http://192.168.93.21/identity/
The default users are: admin and demo
The password: secret

Services are running under systemd unit files.
For more information see: 
https://docs.openstack.org/devstack/latest/systemd.html

DevStack Version: 2024.1
Change: f4f09416ca8de09820b52599c7bcd2911b407466 Merge "Drop unused environments for TripleO and heat agents" 2024-03-19 18:55:14 +0000
OS Version: Ubuntu 22.04 jammy

修改网卡配置，增加bridges部分，使网桥开机自动启动

stack@devstack21:~/devstack$ sudo vim /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:
  ethernets:
    ens33:
      addresses:
      - 192.168.93.21/24
      nameservers:
        addresses:
        - 223.5.5.5
        - 223.6.6.6
        search: []
      routes:
        - to: default
          via: 192.168.93.2
    ens37:
      dhcp4: false
  bridges:
    br-ex:
      openvswitch: {}
      dhcp4: false
  version: 2

应用网卡配置

stack@devstack21:~/devstack$ sudo netplan apply 
stack@devstack21:~/devstack$

查看网卡信息

stack@devstack21:~/devstack$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:b0:da:db brd ff:ff:ff:ff:ff:ff
    altname enp2s1
    inet 192.168.93.21/24 brd 192.168.93.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:feb0:dadb/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000
    link/ether 00:0c:29:b0:da:e5 brd ff:ff:ff:ff:ff:ff
    altname enp2s5
    inet6 fe80::20c:29ff:feb0:dae5/64 scope link 
       valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 22:0d:02:9f:90:1d brd ff:ff:ff:ff:ff:ff
5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether f2:ad:f1:af:54:9a brd ff:ff:ff:ff:ff:ff
6: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:f2:bf:a4 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 00:0c:29:b0:da:e5 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:feb0:dae5/64 scope link 
       valid_lft forever preferred_lft forever

查看网桥信息

stack@devstack21:~/devstack$ sudo ovs-vsctl show
8fc32620-e49f-4e8a-87a1-675d4185e1aa
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-int
        fail_mode: secure
        datapath_type: system
        Port br-int
            Interface br-int
                type: internal
        Port patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575
            Interface patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575
                type: patch
                options: {peer=patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-int}
    Bridge br-ex
        fail_mode: standalone
        Port br-ex
            Interface br-ex
                type: internal
        Port ens37
            Interface ens37
        Port patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-int
            Interface patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-int
                type: patch
                options: {peer=patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575}
    ovs_version: "2.17.9"

此时已经可以登录dashboard，并创建虚拟机，默认账号为admin/secret。

http://192.168.93.21/

安装计算节点

以下操作在计算节点执行。

从neutron文件夹下复制计算节点示例配置文件

stack@devstack22:~$ pwd
/opt/stack
stack@devstack22:~$ cd devstack
stack@devstack22:~$ egrep -v "^#|^$" ../neutron/devstack/ovn-compute-local.conf.sample > ./local.conf

配置文件修改后如下：

cat >local.conf<<'EOF'
[[local|localrc]]
# git_base
GIT_BASE="https://github.com"
#GIT_BASE=https://opendev.org
#GIT_BASE=http://git.trystack.cn

# password
ADMIN_PASSWORD=secret
DATABASE_PASSWORD=$ADMIN_PASSWORD
RABBIT_PASSWORD=$ADMIN_PASSWORD
SERVICE_PASSWORD=$ADMIN_PASSWORD
SERVICE_TOKEN=$ADMIN_PASSWORD

# OVN
Q_AGENT=ovn
NEUTRON_AGENT=$Q_AGENT
Q_ML2_PLUGIN_MECHANISM_DRIVERS=ovn,logger
Q_ML2_PLUGIN_TYPE_DRIVERS=local,flat,vlan,geneve
Q_ML2_TENANT_NETWORK_TYPE="geneve"
LOGFILE=$DEST/logs/stack.sh.log
enable_plugin neutron https://opendev.org/openstack/neutron
disable_all_services
enable_service n-cpu
enable_service placement-client
enable_service ovn-controller
enable_service q-ovn-metadata-agent
SERVICE_HOST=192.168.93.21
RABBIT_HOST=$SERVICE_HOST
Q_HOST=$SERVICE_HOST
OVN_SB_REMOTE=tcp:$SERVICE_HOST:6642
OVN_BUILD_MODULES=True
HOST_IP=192.168.93.22
NOVA_VNC_ENABLED=True
NOVNCPROXY_URL=http://$SERVICE_HOST:6080/vnc_lite.html
VNCSERVER_LISTEN=$HOST_IP
VNCSERVER_PROXYCLIENT_ADDRESS=$VNCSERVER_LISTEN

## Neutron options
Q_USE_SECGROUP=True
FLOATING_RANGE="192.168.72.0/24"
Q_FLOATING_ALLOCATION_POOL=start=192.168.72.110,end=192.168.72.120
PUBLIC_NETWORK_GATEWAY="192.168.72.8"
PUBLIC_INTERFACE=ens37

# Open vSwitch provider networking configuration
Q_USE_PROVIDERNET_FOR_PUBLIC=True
OVS_PHYSICAL_BRIDGE=br-ex
PUBLIC_BRIDGE=br-ex
OVS_BRIDGE_MAPPINGS=public:br-ex
EOF

主要配置参数说明：

HOST_IP: 本机IP地址
SERVICE_HOST：必须是控制节点IP地址

安装计算节点

$ ./stack.sh

确认安装完成

......
=========================
DevStack Component Timing
 (times are in seconds)  
=========================
wait_for_service       7
async_wait             0
osc                    0
apt-get              264
pip_install          103
apt-get-update         1
run_process            4
git_timed             45
-------------------------
Unaccounted time     100
=========================
Total runtime        524

=================
 Async summary
=================
 Time spent in the background minus waits: 1 sec
 Elapsed time: 524 sec
 Time if we did everything serially: 525 sec
 Speedup:  1.00191



This is your host IP address: 192.168.93.22
This is your host IPv6 address: ::1

Services are running under systemd unit files.
For more information see: 
https://docs.openstack.org/devstack/latest/systemd.html

DevStack Version: 2024.1
Change: f4f09416ca8de09820b52599c7bcd2911b407466 Merge "Drop unused environments for TripleO and heat agents" 2024-03-19 18:55:14 +0000
OS Version: Ubuntu 22.04 jammy

修改网卡配置，增加bridges部分，使网桥开机自动启动

stack@devstack22:~/devstack$ sudo vim /etc/netplan/00-installer-config.yaml
# This is the network config written by 'subiquity'
network:
  ethernets:
    ens33:
      addresses:
      - 192.168.93.22/24
      nameservers:
        addresses:
        - 223.5.5.5
        - 223.6.6.6
        search: []
      routes:
        - to: default
          via: 192.168.93.2
    ens37:
      dhcp4: false
  bridges:
    br-ex:
      openvswitch: {}
      dhcp4: false
  version: 2

应用网卡配置

stack@devstack22:~/devstack$ sudo netplan apply 
stack@devstack22:~/devstack$

查看网卡信息

stack@devstack22:~/devstack$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 00:0c:29:31:a7:2c brd ff:ff:ff:ff:ff:ff
    altname enp2s1
    inet 192.168.93.22/24 brd 192.168.93.255 scope global ens33
       valid_lft forever preferred_lft forever
    inet6 fe80::20c:29ff:fe31:a72c/64 scope link 
       valid_lft forever preferred_lft forever
3: ens37: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master ovs-system state UP group default qlen 1000
    link/ether 00:0c:29:31:a7:36 brd ff:ff:ff:ff:ff:ff
    altname enp2s5
    inet6 fe80::20c:29ff:fe31:a736/64 scope link 
       valid_lft forever preferred_lft forever
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 22:0d:02:9f:90:1d brd ff:ff:ff:ff:ff:ff
5: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 06:0d:1b:71:58:0e brd ff:ff:ff:ff:ff:ff
6: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:42:16:35 brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
7: br-ex: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 00:0c:29:31:a7:36 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::20c:29ff:fe31:a736/64 scope link 
       valid_lft forever preferred_lft forever
8: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
    link/ether 1e:dd:eb:ff:7a:66 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::f4be:3fff:fe9b:6d89/64 scope link 
       valid_lft forever preferred_lft forever

查看网桥信息

stack@devstack22:~/devstack$ sudo ovs-vsctl show
15b21aff-829b-4c9f-8ef6-1fa372eddb3a
    Manager "ptcp:6640:127.0.0.1"
        is_connected: true
    Bridge br-ex
        fail_mode: standalone
        Port br-ex
            Interface br-ex
                type: internal
        Port ens37
            Interface ens37
        Port patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-int
            Interface patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-int
                type: patch
                options: {peer=patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575}
    Bridge br-int
        fail_mode: secure
        datapath_type: system
        Port patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575
            Interface patch-br-int-to-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575
                type: patch
                options: {peer=patch-provnet-4ff27886-1c19-4ba4-bd52-05e6c6524575-to-br-int}
        Port ovn-2018dc-0
            Interface ovn-2018dc-0
                type: geneve
                options: {csum="true", key=flow, remote_ip="192.168.93.21"}
                bfd_status: {diagnostic="No Diagnostic", flap_count="1", forwarding="true", remote_diagnostic="Neighbor Signaled Session Down", remote_state=up, state=up}
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.17.9"

节点发现

以下操作在控制节点执行。

切换到管理员用户

stack@devstack21:~/devstack$ . openrc admin admin
WARNING: setting legacy OS_TENANT_NAME to support cli tools.
stack@devstack21:~/devstack$

执行节点发现脚本

stack@devstack21:~/devstack$ pwd
/opt/stack/devstack
stack@devstack21:~/devstack$ ./tools/discover_hosts.sh

执行结果如下

stack@devstack21:~/devstack$ ./tools/discover_hosts.sh 
INFO dbcounter [None req-1bb45b78-fd14-4fda-bd70-8cb20326bb8e None None] Registered counter for database nova_api
DEBUG dbcounter [-] [131423] Writer thread running {{(pid=131423) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}}
Found 2 cell mappings.
Skipping cell0 since it does not contain hosts.
Getting computes from cell 'cell1': ac941876-1395-4846-befe-ee823eb14d5e
INFO dbcounter [None req-1bb45b78-fd14-4fda-bd70-8cb20326bb8e None None] Registered counter for database nova_cell1
DEBUG dbcounter [-] [131423] Writer thread running {{(pid=131423) stat_writer /opt/stack/data/venv/lib/python3.10/site-packages/dbcounter.py:102}}
Found 0 unmapped computes in cell: ac941876-1395-4846-befe-ee823eb14d5e
stack@devstack21:~/devstack$

查看新发现的hypervisor

stack@devstack21:~/devstack$ openstack hypervisor list
+--------------------------------------+---------------------+-----------------+---------------+-------+
| ID                                   | Hypervisor Hostname | Hypervisor Type | Host IP       | State |
+--------------------------------------+---------------------+-----------------+---------------+-------+
| be439833-6735-4f3e-9ddd-3e37f768e18d | devstack21          | QEMU            | 192.168.93.21 | up    |
| a2b491c5-8124-4c49-b279-e327d98bb412 | devstack22          | QEMU            | 192.168.93.22 | up    |
+--------------------------------------+---------------------+-----------------+---------------+-------+
stack@devstack21:~/devstack$

还可以查看 OVN 和 OVS 以查看第二个主机已出现。例如，OVN_Southbound 数据库的 Chassis 表中将有第二个条目。可以使用 ovn-sbctl 实用程序列出chassis、其配置以及绑定到每个chassis的端口：

stack@devstack21:~/devstack$ sudo ovn-sbctl show
Chassis "2018dc69-862f-45d9-ad7d-0fc4dc5d08c4"
    hostname: devstack21
    Encap geneve
        ip: "192.168.93.21"
        options: {csum="true"}
    Port_Binding cr-lrp-7ff9bc36-6877-44c2-aee9-d9d8eb22c8e0
Chassis "1e889c0c-2509-4cc5-ad6e-f9d3bdfb5c40"
    hostname: devstack22
    Encap geneve
        ip: "192.168.93.22"
        options: {csum="true"}

还可以看到创建到另一个计算节点的隧道：

stack@devstack21:~/devstack$ sudo ovs-vsctl show
......
    Bridge br-int
        fail_mode: secure
        datapath_type: system
        Port br-int
            Interface br-int
                type: internal
        Port ovn-1e889c-0
            Interface ovn-1e889c-0
                type: geneve
                options: {csum="true", key=flow, remote_ip="192.168.93.22"}
......

两节点openstack已经部署完成，你可以通过openstack CLI或登录dashboard创建虚拟机实例进一步测试。

创建虚拟机

在本节中，我们将使用openstack CLI逐步完成创建两个虚拟机的步骤，这些虚拟机具有连接到 private Neutron 网络的虚拟 NIC。

DevStack 默认使用 libvirt 作为 Nova 后端。如果 KVM 可用，则会使用它。否则，它只会运行 qemu 模拟来宾。这对于我们的测试来说非常合适，因为我们只需要这些虚拟机能够发送和接收少量流量，因此性能并不是很重要。

虚拟机创建流程

1、切换到demo用户

stack@devstack21:~/devstack$ pwd
/opt/stack/devstack
stack@devstack21:~/devstack$ . openrc demo demo
WARNING: setting legacy OS_TENANT_NAME to support cli tools.
stack@devstack21:~/devstack$

2、获取网络 UUID。

查看可用的网络，devstack默认为我们创建了三个网络，我们将使用 private 租户网络创建虚拟机

stack@devstack21:~/devstack$ openstack network list
+--------------------------------------+---------+----------------------------------------------------------------------------+
| ID                                   | Name    | Subnets                                                                    |
+--------------------------------------+---------+----------------------------------------------------------------------------+
| 6211312f-ab8e-4346-bc58-aed8a63d4f62 | public  | 0cf53da9-2c68-4b7d-8043-8d682fe34b23, 8a1dbf88-3caa-4a88-afa4-6fb6823cb224 |
| 8b8cdb32-c009-49e1-88d4-3d58198e6d6c | private | 24bfad1c-46ab-45a8-b607-da0ed8ea0b3c, 912dae4c-691a-44be-afb1-252c19ebfe24 |
| 98a54015-c7ac-4825-80a8-b3364d41db43 | shared  | 0e754263-61ee-496a-8187-3103af900846                                       |
+--------------------------------------+---------+----------------------------------------------------------------------------+

首先从之前的 openstack network list 输出中获取 private 网络的 UUID 并将其保存：

PRIVATE_NET_ID=$(openstack network show private -c id -f value)

3、创建 SSH 密钥对。

接下来在 Nova 中创建 SSH 密钥对。稍后，当我们启动虚拟机时，我们会要求将公钥放入虚拟机中，以便我们可以通过 SSH 访问它。

openstack keypair create demo > id_rsa_demo
chmod 600 id_rsa_demo

查看我们创建的keypair

stack@devstack21:~/devstack$ openstack keypair list
+------+-------------------------------------------------+------+
| Name | Fingerprint                                     | Type |
+------+-------------------------------------------------+------+
| demo | dc:8b:e9:ce:37:c9:bd:16:57:df:c0:20:dc:8c:64:d0 | ssh  |
+------+-------------------------------------------------+------+
stack@devstack21:~/devstack$

3、选择一种flavor，devstack默认为我们创建了多种flavor。

我们需要为这些测试虚拟机提供最少的资源，因此 m1.nano 风格就足够了。

stack@devstack21:~/devstack$ openstack flavor list
+----+-----------+-------+------+-----------+-------+-----------+
| ID | Name      |   RAM | Disk | Ephemeral | VCPUs | Is Public |
+----+-----------+-------+------+-----------+-------+-----------+
| 1  | m1.tiny   |   512 |    1 |         0 |     1 | True      |
| 2  | m1.small  |  2048 |   20 |         0 |     1 | True      |
| 3  | m1.medium |  4096 |   40 |         0 |     2 | True      |
| 4  | m1.large  |  8192 |   80 |         0 |     4 | True      |
| 42 | m1.nano   |   128 |    1 |         0 |     1 | True      |
| 5  | m1.xlarge | 16384 |  160 |         0 |     8 | True      |
| 84 | m1.micro  |   192 |    1 |         0 |     1 | True      |
| c1 | cirros256 |   256 |    1 |         0 |     1 | True      |
| d1 | ds512M    |   512 |    5 |         0 |     1 | True      |
| d2 | ds1G      |  1024 |   10 |         0 |     1 | True      |
| d3 | ds2G      |  2048 |   10 |         0 |     2 | True      |
| d4 | ds4G      |  4096 |   20 |         0 |     4 | True      |
+----+-----------+-------+------+-----------+-------+-----------+

$ FLAVOR_ID=$(openstack flavor show m1.nano -c id -f value)

4、选择一个镜像，devstack 默认已经为我们创建了一个cirros镜像

DevStack 默认导入 CirrOS 镜像，这非常适合我们的测试。这是一个非常小的测试图像。

stack@devstack21:~/devstack$ openstack image list
+--------------------------------------+--------------------------+--------+
| ID                                   | Name                     | Status |
+--------------------------------------+--------------------------+--------+
| df855e1f-4752-4d3c-a81b-6dc5ebfa96ab | cirros-0.6.2-x86_64-disk | active |
+--------------------------------------+--------------------------+--------+

$ IMAGE_ID=$(openstack image list -c ID -f value)

5、设置安全规则，以便我们可以访问接下来要启动的虚拟机。

默认情况下，DevStack 不允许用户访问虚拟机，要启用它，我们需要添加一条规则。我们将允许 ICMP 和 SSH。

openstack security group rule create --ingress --ethertype IPv4 --dst-port 22 --protocol tcp default
openstack security group rule create --ingress --ethertype IPv4 --protocol ICMP default

查看创建的安全组规则

$ openstack security group rule list
+--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+
| ID                                   | IP Protocol | IP Range  | Port Range | Remote Security Group                | Security Group                       |
+--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+
...
| ade97198-db44-429e-9b30-24693d86d9b1 | tcp         | 0.0.0.0/0 | 22:22      | None                                 | a47b14da-5607-404a-8de4-3a0f1ad3649c |
| d0861a98-f90e-4d1a-abfb-827b416bc2f6 | icmp        | 0.0.0.0/0 |            | None                                 | a47b14da-5607-404a-8de4-3a0f1ad3649c |
...
+--------------------------------------+-------------+-----------+------------+--------------------------------------+--------------------------------------+

6、启动一些虚拟机。

现在我们将启动两个虚拟机。我们将它们命名为 test1 和 test2 。

stack@devstack21:~/devstack$ openstack server create --nic net-id=$PRIVATE_NET_ID --flavor $FLAVOR_ID --image $IMAGE_ID --key-name demo test1
+--------------------------------------+-----------------------------------------------------------------+
| Field                                | Value                                                           |
+--------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                          |
| OS-EXT-AZ:availability_zone          |                                                                 |
| OS-EXT-STS:power_state               | NOSTATE                                                         |
| OS-EXT-STS:task_state                | scheduling                                                      |
| OS-EXT-STS:vm_state                  | building                                                        |
| OS-SRV-USG:launched_at               | None                                                            |
| OS-SRV-USG:terminated_at             | None                                                            |
| accessIPv4                           |                                                                 |
| accessIPv6                           |                                                                 |
| addresses                            |                                                                 |
| adminPass                            | Y5j83PELiYKs                                                    |
| config_drive                         |                                                                 |
| created                              | 2024-03-24T04:42:57Z                                            |
| flavor                               | m1.nano (42)                                                    |
| hostId                               |                                                                 |
| id                                   | 88ea2c47-7fe9-460b-b98a-0e4ebf773c98                            |
| image                                | cirros-0.6.2-x86_64-disk (df855e1f-4752-4d3c-a81b-6dc5ebfa96ab) |
| key_name                             | demo                                                            |
| name                                 | test1                                                           |
| os-extended-volumes:volumes_attached | []                                                              |
| progress                             | 0                                                               |
| project_id                           | 6525123a5d8d4f048d51edf1ed18f06b                                |
| properties                           |                                                                 |
| security_groups                      | name='default'                                                  |
| status                               | BUILD                                                           |
| updated                              | 2024-03-24T04:42:57Z                                            |
| user_id                              | 55e4cf4eec0f48efa3ffa182046af847                                |
+--------------------------------------+-----------------------------------------------------------------+
stack@devstack21:~/devstack$ 

stack@devstack21:~/devstack$ openstack server create --nic net-id=$PRIVATE_NET_ID --flavor $FLAVOR_ID --image $IMAGE_ID --key-name demo test2
+--------------------------------------+-----------------------------------------------------------------+
| Field                                | Value                                                           |
+--------------------------------------+-----------------------------------------------------------------+
| OS-DCF:diskConfig                    | MANUAL                                                          |
| OS-EXT-AZ:availability_zone          |                                                                 |
| OS-EXT-STS:power_state               | NOSTATE                                                         |
| OS-EXT-STS:task_state                | scheduling                                                      |
| OS-EXT-STS:vm_state                  | building                                                        |
| OS-SRV-USG:launched_at               | None                                                            |
| OS-SRV-USG:terminated_at             | None                                                            |
| accessIPv4                           |                                                                 |
| accessIPv6                           |                                                                 |
| addresses                            |                                                                 |
| adminPass                            | m3dhpx8meFHh                                                    |
| config_drive                         |                                                                 |
| created                              | 2024-03-24T04:43:35Z                                            |
| flavor                               | m1.nano (42)                                                    |
| hostId                               |                                                                 |
| id                                   | 8a32e126-7f97-4e1a-9a06-059867643785                            |
| image                                | cirros-0.6.2-x86_64-disk (df855e1f-4752-4d3c-a81b-6dc5ebfa96ab) |
| key_name                             | demo                                                            |
| name                                 | test2                                                           |
| os-extended-volumes:volumes_attached | []                                                              |
| progress                             | 0                                                               |
| project_id                           | 6525123a5d8d4f048d51edf1ed18f06b                                |
| properties                           |                                                                 |
| security_groups                      | name='default'                                                  |
| status                               | BUILD                                                           |
| updated                              | 2024-03-24T04:43:34Z                                            |
| user_id                              | 55e4cf4eec0f48efa3ffa182046af847                                |
+--------------------------------------+-----------------------------------------------------------------+

两个虚拟机启动后，它们的状态将为 ACTIVE ：

stack@devstack22:~/devstack$ openstack server list
+--------------------------------------+-------+--------+--------------------------------------------------------+--------------------------+---------+
| ID                                   | Name  | Status | Networks                                               | Image                    | Flavor  |
+--------------------------------------+-------+--------+--------------------------------------------------------+--------------------------+---------+
| 8a32e126-7f97-4e1a-9a06-059867643785 | test2 | ACTIVE | private=10.0.0.54, fd66:a9a:3259:0:f816:3eff:fe5e:4f23 | cirros-0.6.2-x86_64-disk | m1.nano |
| 88ea2c47-7fe9-460b-b98a-0e4ebf773c98 | test1 | ACTIVE | private=10.0.0.36, fd66:a9a:3259:0:f816:3eff:fe6f:4b92 | cirros-0.6.2-x86_64-disk | m1.nano |
+--------------------------------------+-------+--------+--------------------------------------------------------+--------------------------+---------+
stack@devstack22:~/devstack$

虚拟机连接

我们可以通过关联公共网络中的浮动 IP 地址来连接到我们的虚拟机。

stack@devstack22:~/devstack$ TEST1_PORT_ID=$(openstack port list --server test1 -c id -f value)
stack@devstack22:~/devstack$ openstack floating ip create --port $TEST1_PORT_ID public
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field               | Value                                                                                                                                                                              |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| created_at          | 2024-03-24T05:26:11Z                                                                                                                                                               |
| description         |                                                                                                                                                                                    |
| dns_domain          |                                                                                                                                                                                    |
| dns_name            |                                                                                                                                                                                    |
| fixed_ip_address    | 10.0.0.36                                                                                                                                                                          |
| floating_ip_address | 192.168.72.119                                                                                                                                                                     |
| floating_network_id | 6211312f-ab8e-4346-bc58-aed8a63d4f62                                                                                                                                               |
| id                  | 8b830932-5b79-4445-8730-2143e1779e54                                                                                                                                               |
| name                | 192.168.72.119                                                                                                                                                                     |
| port_details        | {'name': '', 'network_id': '8b8cdb32-c009-49e1-88d4-3d58198e6d6c', 'mac_address': 'fa:16:3e:6f:4b:92', 'admin_state_up': True, 'status': 'ACTIVE', 'device_id':                    |
|                     | '88ea2c47-7fe9-460b-b98a-0e4ebf773c98', 'device_owner': 'compute:nova'}                                                                                                            |
| port_forwardings    | []                                                                                                                                                                                 |
| port_id             | 5fc6a820-8b4e-404b-899d-b57090708efd                                                                                                                                               |
| project_id          | 6525123a5d8d4f048d51edf1ed18f06b                                                                                                                                                   |
| qos_policy_id       | None                                                                                                                                                                               |
| revision_number     | 0                                                                                                                                                                                  |
| router_id           | c0e52c09-8791-4a4e-9c82-9cd40d27a65e                                                                                                                                               |
| status              | DOWN                                                                                                                                                                               |
| subnet_id           | None                                                                                                                                                                               |
| tags                | []                                                                                                                                                                                 |
| updated_at          | 2024-03-24T05:26:11Z                                                                                                                                                               |
+---------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
stack@devstack22:~/devstack$

现在应该能够通过其浮动 IP 地址连接到虚拟机。首先，ping 地址。

stack@devstack21:~/devstack$ ping 192.168.72.119 -c 3
PING 192.168.72.119 (192.168.72.119) 56(84) bytes of data.
64 bytes from 192.168.72.119: icmp_seq=1 ttl=128 time=2.43 ms
64 bytes from 192.168.72.119: icmp_seq=2 ttl=128 time=1.75 ms
64 bytes from 192.168.72.119: icmp_seq=3 ttl=128 time=1.03 ms

--- 192.168.72.119 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 1.031/1.735/2.425/0.569 ms
stack@devstack21:~/devstack$

现在通过 SSH 连接到虚拟机：

stack@devstack21:~/devstack$ ssh -i id_rsa_demo cirros@192.168.72.119 hostname
test1
stack@devstack21:~/devstack$

登录dashboard

切换到demo项目，查看虚拟机管理器
在这里插入图片描述
查看test1实例位于devstack22节点上

查看test2实例位于devstack21节点上

查看网络拓扑
在这里插入图片描述

willops

关注

28
点赞
踩
15

收藏

觉得还不错? 一键收藏
1
评论
DevStack 部署 OpenStack 多节点

DevStack 支持OpenStack多节点部署，下面以一个控制节点和一个计算节点为例，介绍多节点多网卡部署流程。
复制链接

扫一扫

专栏目录

DevStack 部署 OpenStack 多节点

DevStack 部署 OpenStack 多节点

节点规划

前置准备

添加 Stack 用户

下载 DevStack

安装控制节点

安装计算节点

节点发现

创建虚拟机

虚拟机创建流程

虚拟机连接

登录dashboard

“相关推荐”对你有帮助么？