Openstack安装Openstack（Juno版本）

最新推荐文章于 2024-08-09 10:53:56 发布

乌托邦2号

最新推荐文章于 2024-08-09 10:53:56 发布

阅读量4.6k

点赞数 1

分类专栏： Linux操作系统文章标签： linux

Linux操作系统专栏收录该内容

91 篇文章 5 订阅

订阅专栏

国内现在做Openstack的培训也开始多起来，不过应该都是采用虚拟机的方式来培训，我个人感觉还是很土的，不能吃自己的狗食。建议日后大家参加Openstack的培训的时候，都问一句，是不是在Openstack下培训Openstack。

上次写过一篇Icehouse版本的。这次Juno版本，基本是一样的过程。不过调整一下顺序。用用户更容易操作。

这次我是使用刻通云平台进行，希望可以做到更加流畅。

（1）基本情况

默认是有一个基础网络。我们还是需要创建一个自己的网络，来满足openstack需求。

整理一下

角色	管理网络	虚拟机通讯网络	外部网络
控制节点	eth0（10.0.0.11）	eth1（192.168.100.11）
网络节点	eth0（10.0.0.21）	eth1（10.0.1.21）	eth2（192.168.100.21）
计算节点	eth0（10.0.0.31）	eth1（10.0.1.21）

文档很清楚，

网络节点，需要3块网卡。
控制节点和网络节点，需要外部网络，就是需要所谓的公网的IP
计算节点是不需要公网IP
所有的虚拟机访问公网，都是需要经过网络节点。
192.168.100.0,就相当于公网的IP地址段

根据上图，我们组建我们自己的网络

创建router
创建管理网络，公网，虚拟机网络（记得按顺序创建）
公网连接router
申请公网IP
把IP绑定在router

（2）控制节点

1、网络搭建

创建一个ubuntu14.04的虚拟机，1core，2G内存，应该就够用了。网络，记得设置固定IP地址

为了登陆vnc，所以选择密码登陆

最后一步

虚拟机就创建完毕。

我们需要给控制节点添加一块网卡，连接到公网

看看拓扑图

vnc登陆

由于虚拟机获得两块网卡，默认网关就需要手工指定。

route add default gw 192.168.100.1

这个时候,你就可以访问外网.

从远程访问虚拟机，可以通过端口映射，或者vpn，这里直接在router上设置端口映射。这样你直接sshrouter的IP地址，就可以访问。

2、设置源

apt-get install ubuntu-cloud-keyring
echo "deb http://ubuntu-cloud.archive.canonical.com/ubuntu" \
  "trusty-updates/juno main" > /etc/apt/sources.list.d/cloudarchive-juno.list

更新

apt-get update && apt-get dist-upgrade

3、NTP服务器

apt-get install -y ntp

4、数据库

apt-get install mariadb-server python-mysqldb

修改/etc/mysql/my.cnf

bind-address = 10.0.0.11

[mysqld]
default-storage-engine = innodb
innodb_file_per_table
collation-server = utf8_general_ci
init-connect = 'SET NAMES utf8'
character-set-server = utf8

重启数据库

service mysql restart

5、消息队列RabbitMQ

apt-get install -y rabbitmq-server

6、keystone

安装

apt-get install -y keystone

设置

创建keystone数据库，都是通过 mysql –u root –p 进入

CREATE DATABASE keystone;
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' IDENTIFIED BY 'KEYSTONE_DBPASS';
GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'%' IDENTIFIED BY 'KEYSTONE_DBPASS';

exit;

删除sqllite数据库

rm /var/lib/keystone/keystone.db

编辑 /etc/keystone/keystone.conf

connection = mysql://keystone:KEYSTONE_DBPASS@10.0.0.11/keystone

[DEFAULT]
admin_token=ADMIN
log_dir=/var/log/keystone

初始化keystone数据库

service keystone restart
keystone-manage db_sync

设置环境变量

export OS_SERVICE_TOKEN=ADMIN
export OS_SERVICE_ENDPOINT=http://10.0.0.11:35357/v2.0

创建管理员权力的用户

keystone user-create --name=admin --pass=admin_pass --email=admin@domain.com
keystone role-create --name=admin
keystone role-create --name=_member_
keystone tenant-create --name=admin --description="Admin Tenant"
keystone user-role-add --user=admin --tenant=admin --role=admin
keystone user-role-add --user=admin --role=_member_ --tenant=admin

创建普通用户

keystone user-create --name=demo --pass=demo_pass --email=demo@domain.com
keystone tenant-create --name=demo --description="Demo Tenant"
keystone user-role-add --user=demo --role=_member_ --tenant=demo

创建 service 租户

keystone tenant-create --name=service --description="Service Tenant"

定义服务的API的endpoint

keystone service-create --name=keystone --type=identity --description="OpenStack Identity"

创建endpoint

keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ identity / {print $2}') \
--publicurl=http://192.168.100.11:5000/v2.0 \
--internalurl=http://10.0.0.11:5000/v2.0 \
--adminurl=http://10.0.0.11:35357/v2.0

检测keystone

通过下面命令检查keystone的初始化是否正常

设置环境变量，创建creds 和 admin_creds 两个文件

cat <<EOF >>/root/creds
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL="http://192.168.100.11:5000/v2.0/"
EOF
cat <<EOF >>/root/admin_creds
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_TENANT_NAME=admin
export OS_AUTH_URL=http://10.0.0.11:35357/v2.0
EOF

检测

先退出ssh，让以前设置的环境变量失效。再登陆。

设置环境变量才能进行下面操作

source creds

这样就可以

root@controller:~# keystone user-list
+----------------------------------+-------+---------+------------------+
|                id                |  name | enabled |      email       |
+----------------------------------+-------+---------+------------------+
| 6f8bcafd62ec4e23ab2be28016829f91 | admin |   True  | admin@domain.com |
| 66713a75b7c14f73a1c5a015241f5826 |  demo |   True  | demo@domain.com  |
+----------------------------------+-------+---------+------------------+
root@controller:~# keystone role-list
+----------------------------------+----------+
|                id                |   name   |
+----------------------------------+----------+
| 9fe2ff9ee4384b1894a90878d3e92bab | _member_ |
| cd8dec7752d24a028f95657556f7573d |  admin   |
+----------------------------------+----------+
root@controller:~# keystone tenant-list
+----------------------------------+---------+---------+
|                id                |   name  | enabled |
+----------------------------------+---------+---------+
| efc81990ab4c433f94573e2e0fcf08c3 |  admin  |   True  |
| be10dc11d4034b389bef8bbcec657f6f |   demo  |   True  |
| cb45c886bc094f65940ba29d79eab8aa | service |   True  |
+----------------------------------+---------+---------+

查看日志

日志在/var/log/keystone/ 下，先清空日志,看看日志是否还有错误信息.

echo "" > /var/log/keystone/keystone-all.log
echo "" > /var/log/keystone/keystone-manage.log
tail  /var/log/keystone/*

7、Glance

Openstack组件安装，都比较类似。

apt-get install -y glance python-glanceclient

创建数据库 mysql –uroot –p

CREATE DATABASE glance;
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' IDENTIFIED BY 'GLANCE_DBPASS';
GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'%' IDENTIFIED BY 'GLANCE_DBPASS';

exit;

keystone创建glance用户和服务

keystone user-create --name=glance --pass=service_pass --email=glance@domain.com
keystone user-role-add --user=glance --tenant=service --role=admin

设置endpoint

keystone service-create --name=glance --type=image --description="OpenStack Image Service"
keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ image / {print $2}') \
--publicurl=http://192.168.100.11:9292 \
--internalurl=http://10.0.0.11:9292 \
--adminurl=http://10.0.0.11:9292

编辑 /etc/glance/glance-api.conf

[database]
connection = mysql://glance:GLANCE_DBPASS@10.0.0.11/glance

[DEFAULT]
rpc_backend = rabbit
rabbit_host = 10.0.0.11

[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
identity_uri = http://10.0.0.11:35357
admin_tenant_name = service
admin_user = glance
admin_password = service_pass

[paste_deploy]
flavor = keystone

编辑 /etc/glance/glance-registry.conf

[database]
# The file name to use with SQLite (string value)
#sqlite_db = /var/lib/glance/glance.sqlite
connection = mysql://glance:GLANCE_DBPASS@10.0.0.11/glance


[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = glance
admin_password = service_pass

[paste_deploy]
flavor = keystone

重启服务

service glance-api restart; service glance-registry restart

初始化glance数据库

glance-manage db_sync

上传镜像

source creds
glance image-create --name "cirros-0.3.2-x86_64" --is-public true \
--container-format bare --disk-format qcow2 \
--location http://cdn.download.cirros-cloud.net/0.3.2/cirros-0.3.2-x86_64-disk.img

查看镜像

# glance image-list
+--------------------------------------+---------------------+-------------+------------------+----------+--------+
| ID                                   | Name                | Disk Format | Container Format | Size     | Status |
+--------------------------------------+---------------------+-------------+------------------+----------+--------+
| d7a6d71d-4222-44f4-82d0-49c14ba19676 | cirros-0.3.2-x86_64 | qcow2       | bare             | 13167616 | active |
+--------------------------------------+---------------------+-------------+------------------+----------+--------+

检测log

root@controller:~# tail /var/log/glance/*
==> /var/log/glance/api.log <==
-09-02 07:07:12.315 2946 WARNING glance.store.base [-] Failed to configure store correctly:
 Store sheepdog could not be configured correctly. Reason:
 Error in store configuration: [Errno 2] No such file or directory Disabling add method.
-09-02 07:07:12.316 2946 WARNING glance.store [-] Deprecated: glance.store.
sheepdog.Store not found in `known_store`. 
Stores need to be explicitly enabled in the configuration file.

你会发现log里有类似的所谓错误，这个不是问题。希望glance改进一下这个地方的log。不然让很多新手很郁闷。

8、Nova

安装软件

apt-get install -y nova-api nova-cert nova-conductor nova-consoleauth \
nova-novncproxy nova-scheduler python-novaclient

创建nova 数据库 mysql –u root –p

CREATE DATABASE nova;
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' IDENTIFIED BY 'NOVA_DBPASS';
GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'%' IDENTIFIED BY 'NOVA_DBPASS';

exit;

配置keystone

keystone user-create --name=nova --pass=service_pass --email=nova@domain.com
keystone user-role-add --user=nova --tenant=service --role=admin

设置endpoint

keystone service-create --name=nova --type=compute --description="OpenStack Compute"
keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ compute / {print $2}') \
--publicurl=http://192.168.100.11:8774/v2/%\(tenant_id\)s \
--internalurl=http://10.0.0.11:8774/v2/%\(tenant_id\)s \
--adminurl=http://10.0.0.11:8774/v2/%\(tenant_id\)s

编辑 /etc/nova/nova.conf

下面是我的nova.conf 文件的全部内容

[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
force_dhcp_release=True
iscsi_helper=tgtadm
libvirt_use_virtio_for_bridges=True
connection_type=libvirt
root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf
verbose=True
ec2_private_dns_show_ip=True
api_paste_config=/etc/nova/api-paste.ini
volumes_path=/var/lib/nova/volumes
enabled_apis=ec2,osapi_compute,metadata

rpc_backend = rabbit
rabbit_host = 10.0.0.11
my_ip = 10.0.0.11
vncserver_listen = 10.0.0.11
vncserver_proxyclient_address = 10.0.0.11
auth_strategy = keystone

[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = service_pass

[database]
connection = mysql://nova:NOVA_DBPASS@10.0.0.11/nova

删除sqlite数据库

rm /var/lib/nova/nova.sqlite

初始化nova数据库

nova-manage db sync

重启nova相关服务

service nova-api restart
service nova-cert restart
service nova-conductor restart
service nova-consoleauth restart
service nova-novncproxy restart
service nova-scheduler restart

检查

# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-cert        controller                           internal         enabled       2014-08-26 14:13:08
nova-consoleauth controller                           internal         enabled       2014-08-26 14:13:08
nova-conductor   controller                           internal         enabled       2014-08-26 14:13:08
nova-scheduler   controller                           internal         enabled       2014-08-26 14:13:08

9、Neutron

控制节点，也是需要安装Neutron server

apt-get install -y neutron-server neutron-plugin-ml2

创建Neutron数据库 mysql –u root –p

CREATE DATABASE neutron;
GRANT ALL PRIVILEGES ON neutron.* TO neutron@'localhost' IDENTIFIED BY 'NEUTRON_DBPASS';
GRANT ALL PRIVILEGES ON neutron.* TO neutron@'%' IDENTIFIED BY 'NEUTRON_DBPASS';

exit;

keystone创建neutron用户和角色

keystone user-create --name=neutron --pass=service_pass --email=neutron@domain.com
keystone user-role-add --user=neutron --tenant=service --role=admin

注册服务和endpoint

keystone service-create --name=neutron --type=network --description="OpenStack Networking"

keystone endpoint-create \
--service-id=$(keystone service-list | awk '/ network / {print $2}') \
--publicurl=http://192.168.100.11:9696 \
--internalurl=http://10.0.0.11:9696 \
--adminurl=http://10.0.0.11:9696

编辑 /etc/neutron/neutron.conf，关键的是nova_admin_tenant_id 需要你手工用命令获得，再填写

keystone tenant-list | awk '/ service / { print $2 }'

#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
core_plugin = ml2

# service_plugins =
# Example: service_plugins = router,firewall,lbaas,vpnaas,metering
service_plugins = router

# auth_strategy = keystone
auth_strategy = keystone

# allow_overlapping_ips = False
allow_overlapping_ips = True

rpc_backend = rabbit

rabbit_host = 10.0.0.11

notification_driver = neutron.openstack.common.notifier.rpc_notifier

# ======== neutron nova interactions ==========
# Send notification to nova when port status is active.
notify_nova_on_port_status_changes = True

# Send notifications to nova when port data (fixed_ips/floatingips) change
# so nova can update it's cache.
notify_nova_on_port_data_changes = True

# URL for connection to nova (Only supports one nova region currently).
nova_url = http://10.0.0.11:8774/v2

# Name of nova region to use. Useful if keystone manages more than one region
# nova_region_name =

# Username for connection to nova in admin context
nova_admin_username = nova

# The uuid of the admin nova tenant
nova_admin_tenant_id = cb45c886bc094f65940ba29d79eab8aa

# Password for connection to nova in admin context.
nova_admin_password = service_pass

# Authorization URL for connection to nova in admin context.
nova_admin_auth_url = http://10.0.0.11:35357/v2.0

[keystone_authtoken]
#auth_host = 127.0.0.1
#auth_port = 35357
#auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
#signing_dir = $state_path/keystone-signing
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = neutron
admin_password = service_pass

[database]
# This line MUST be changed to actually run the plugin.
# Example:
# connection = mysql://root:pass@127.0.0.1:3306/neutron
# Replace 127.0.0.1 above with the IP address of the database used by the
# main neutron server. (Leave it as is if the database runs on this host.)
#connection = sqlite:var/lib/neutron/neutron.sqlite
connection = mysql://neutron:NEUTRON_DBPASS@10.0.0.11/neutron

编辑 /etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]
type_drivers = gre
tenant_network_types = gre
mechanism_drivers = openvswitch

[ml2_type_gre]
tunnel_id_ranges = 1:1000

[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True

编辑/etc/nova/nova.conf, 让nova支持neutron，在[DEFAULT] 添加

network_api_class=nova.network.neutronv2.api.API
neutron_url=http://10.0.0.11:9696
neutron_auth_strategy=keystone
neutron_admin_tenant_name=service
neutron_admin_username=neutron
neutron_admin_password=service_pass
neutron_admin_auth_url=http://10.0.0.11:35357/v2.0
libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver
firewall_driver=nova.virt.firewall.NoopFirewallDriver
security_group_api=neutron

重启nova服务

service nova-api restart
service nova-scheduler restart
service nova-conductor restart

这里面有一个bug，需要修复 http://www.tuicool.com/articles/vmaiiua

neutron-db-manage --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugins/ml2/ml2_conf.ini upgrade juno

重启neutron服务

service neutron-server restart

查看log

root@controller:~# tail -f /var/log/neutron/*
-09-02 07:27:53.950 5373 WARNING neutron.api.extensions [-] Extension fwaas not supported by any of loaded plugins
-09-02 07:27:53.952 5373 WARNING neutron.api.extensions [-] Extension flavor not supported by any of loaded plugins
-09-02 07:27:53.962 5373 WARNING neutron.api.extensions [-] Extension lbaas_agent_scheduler not supported by any of loaded plugins
-09-02 07:27:53.967 5373 WARNING neutron.api.extensions [-] Extension lbaas not supported by any of loaded plugins
-09-02 07:27:53.969 5373 WARNING neutron.api.extensions [-] Extension metering not supported by any of loaded plugins
-09-02 07:27:53.973 5373 WARNING neutron.api.extensions [-] Extension port-security not supported by any of loaded plugins
-09-02 07:27:53.977 5373 WARNING neutron.api.extensions [-] Extension routed-service-insertion not supported by any of loaded plugins

日志里显示找不到插件，这都是正常的。

10、Horizon

Dashboard的安装，倒是比较简单，不需要创建数据库。

apt-get install -y apache2 memcached libapache2-mod-wsgi openstack-dashboard

编辑 /etc/openstack-dashboard/local_settings.py

#ALLOWED_HOSTS = ['horizon.example.com', ]
ALLOWED_HOSTS = ['localhost','192.168.100.11']

#OPENSTACK_HOST = "127.0.0.1"
OPENSTACK_HOST = "10.0.0.11"

重启apache服务

service apache2 restart; service memcached restart

这个时候，你可以通过 http://192.168.100.11/horizon

看到登录界面，应该是无法登录。

安装Openstack client端

在控制节点装上Openstack的client端，这样会方便很多，很多Neutron的操作，你都可以进行

apt-get -y install python-openstackclient

（3）网络节点

看图理解的更好，这图来自redhat的官方文档。

网络节点需要3块网卡。经常有朋友问，1块网卡是否可以。其实1块网卡肯定也是可以的，不过不利于大家理解。不过大家都很难找到3块网卡的机器，所以在IaaS下来测试，就方便很多。

创建一个虚拟机，名字为：network, 删除网卡，并且添加3块网卡。ssh到虚拟机上，默认是无法访问外网的，原因也很简单，没有默认路由，手工添加默认路由就可以。

由于网络节点,比较特殊,我们需要把网卡的Ip设置成固定 /etc/netwrok/interface

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# Source interfaces
# Please check /etc/network/interfaces.d before changing this file
# as interfaces may have been defined in /etc/network/interfaces.d
# NOTE: the primary ethernet device is defined in
# /etc/network/interfaces.d/eth0
# See LP: #1262951
#source /etc/network/interfaces.d/*.cfg
# The management network interface
  auto eth0
  iface eth0 inet static
  address 10.0.0.21
  netmask 255.255.255.0

# VM traffic interface
  auto eth1
  iface eth1 inet static
  address 10.0.1.21
  netmask 255.255.255.0

# The public network interface
 auto eth2
 iface eth2 inet static
 address 192.168.100.21
 netmask 255.255.255.0
 gateway 192.168.100.1
 dns-nameservers 114.114.114.114

设置完毕，重启虚拟机。

这个时候，你就可以访问外网，安装包。

apt-get update -y && apt-get upgrade -y && apt-get dist-upgrade

同步时间

apt-get install -y ntp

编辑 /etc/ntp.conf

server 10.0.0.11

重启NTP服务

service ntp restart

安装基础组件

apt-get install -y vlan bridge-utils

编辑 /etc/sysctl.conf

net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0

检测

sysctl -p

安装Neutron组件

apt-get install -y neutron-plugin-ml2 neutron-plugin-openvswitch-agent \
dnsmasq neutron-l3-agent neutron-dhcp-agent

编辑 /etc/neutron/neutron.conf , 这里修改的内容，比控制节点少很多。

#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
core_plugin = ml2

# service_plugins =
# Example: service_plugins = router,firewall,lbaas,vpnaas,metering
service_plugins = router

# The strategy to be used for auth.
# Supported values are 'keystone'(default), 'noauth'.
auth_strategy = keystone

allow_overlapping_ips = True

rpc_backend = neutron.openstack.common.rpc.impl_kombu

rabbit_host = 10.0.0.11

[keystone_authtoken]
#auth_host = 127.0.0.1
#auth_port = 35357
#auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
#signing_dir = $state_path/keystone-signing
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = neutron
admin_password = service_pass

编辑 /etc/neutron/l3_agent.ini

interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
use_namespaces = True

编辑 /etc/neutron/dhcp_agent.ini

interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver
dhcp_driver = neutron.agent.linux.dhcp.Dnsmasq
use_namespaces = True

编辑 /etc/neutron/metadata_agent.ini

auth_url = http://10.0.0.11:5000/v2.0
auth_region = regionOne

admin_tenant_name = service
admin_user = neutron
admin_password = service_pass
nova_metadata_ip = 10.0.0.11
metadata_proxy_shared_secret = helloOpenStack

登录控制节点，修改 /etc/nova.conf 在[DEFAULT] 加入下面内容

service_neutron_metadata_proxy = true
neutron_metadata_proxy_shared_secret = helloOpenStack

重启nova api服务

service nova-api restart

编辑 /etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]
type_drivers = gre
tenant_network_types = gre
mechanism_drivers = openvswitch

[ml2_type_gre]
tunnel_id_ranges = 1:1000

[ovs]
local_ip = 10.0.1.21
tunnel_type = gre
enable_tunneling = True

[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True

重启openvswitch

service openvswitch-switch restart

创建br-ex

创建br-ex连接外网，这个不太好理解，看图

大概意思是：我们创建一个bridge br-ex，把br-ex绑定在eth2下，eth2是连接到公网的路由器上的。

ovs-vsctl add-br br-ex
ovs-vsctl add-port br-ex eth2

下面内容是我操作的结果,大家慢慢理解.

编辑 /etc/network/interfaces

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# Source interfaces
# Please check /etc/network/interfaces.d before changing this file
# as interfaces may have been defined in /etc/network/interfaces.d
# NOTE: the primary ethernet device is defined in
# /etc/network/interfaces.d/eth0
# See LP: #1262951
#source /etc/network/interfaces.d/*.cfg
# The management network interface
  auto eth0
  iface eth0 inet static
  address 10.0.0.21
  netmask 255.255.255.0

# VM traffic interface
  auto eth1
  iface eth1 inet static
  address 10.0.1.21
  netmask 255.255.255.0

# The public network interface
# auto eth2
# iface eth2 inet static
# address 192.168.100.21
# netmask 255.255.255.0
# gateway 192.168.100.1
# dns-nameservers 114.114.114.114

auto eth2
iface eth2 inet manual
up ifconfig $IFACE 0.0.0.0 up
up ip link set $IFACE promisc on
down ip link set $IFACE promisc off
down ifconfig $IFACE down

auto br-ex
iface br-ex inet static
address 192.168.100.21
netmask 255.255.255.0
gateway 192.168.100.1
dns-nameservers 114.114.114.114

重启虚拟机

替换br-ex和eth2的mac地址

由于网络的限制，目前192.168.100.21和192.168.100.11是无法通讯的，原因是因为出于安全的考虑，对网络访问的mac地址和ip地址做了绑定和限制。

通过ifconfig 查看网卡的mac地址，通过命令，把mac地址互换。

br-ex mac 地址 c2:32:7d:cf:9d:4
eth2 mac地址 fa:16:3e:80:5d:e6

ip link set eth2 addr c2:32:7d:cf:9d:43
ip link set br-ex addr fa:16:3e:80:5d:e6

这个时候，外部网络的IP就可以互相访问。这些修改是临时性的，如果重启neutron服务，mac地址就会恢复。不过我们实验不需要重启服务。这里提供的是临时的方法，后面有彻底解决问题的办法。

设置环境变量

cat <<EOF >>/root/creds
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=admin_pass
export OS_AUTH_URL="http://192.168.100.11:5000/v2.0/"
EOF

这样你就可以看到安装的agent

source creds
neutron agent-list

# neutron agent-list
+--------------------------------------+--------------------+---------+-------+----------------+
| id                                   | agent_type         | host    | alive | admin_state_up |
+--------------------------------------+--------------------+---------+-------+----------------+
| 3a80d2ea-bcf6-4835-b125-55144948024c | Open vSwitch agent | network |    | True           |
| 4219dd20-c4fd-4586-b2fc-c81bec0015d6 | L3 agent           | network |    | True           |
| e956687f-a658-4226-a34f-368da61e9e44 | Metadata agent     | network |    | True           |
| f3e841f8-b803-4134-9ba6-3152c3db5592 | DHCP agent         | network |    | True           |
+--------------------------------------+--------------------+---------+-------+----------------+

（4）计算节点

创建一个虚拟机，名字为：compute1, 删除网卡，并且添加2块网卡。ssh到虚拟机上.

计算节点默认是不需要接公网，不过由于我需要安装包，必须联网，所以你可以创建完虚拟机后，给虚拟机连接到外部网络，装完系统后，再断开就可以。

route add default gw 192.168.100.1

这个时候，你就可以访问外网，安装包。

apt-get update -y && apt-get upgrade -y && apt-get dist-upgrade

同步时间

apt-get install -y ntp

编辑 /etc/ntp.conf

server 10.0.0.11

重启NTP服务

service ntp restart

安装kvm套件

apt-get install -y kvm libvirt-bin pm-utils

安装计算节点组件

apt-get install -y nova-compute-kvm python-guestfs

让内核只读

dpkg-statoverride  --update --add root root 0644 /boot/vmlinuz-$(uname -r)

创建脚本 /etc/kernel/postinst.d/statoverride

#!/bin/sh
version="$1"
# passing the kernel version is required
[ -z "${version}" ] && exit 0
dpkg-statoverride --update --add root root 0644 /boot/vmlinuz-${version}

允许运行

chmod +x /etc/kernel/postinst.d/statoverride

编辑 /etc/nova/nova.conf 文件，添加下面内容

[DEFAULT]
dhcpbridge_flagfile=/etc/nova/nova.conf
dhcpbridge=/usr/bin/nova-dhcpbridge
logdir=/var/log/nova
state_path=/var/lib/nova
lock_path=/var/lock/nova
force_dhcp_release=True
iscsi_helper=tgtadm
libvirt_use_virtio_for_bridges=True
connection_type=libvirt
root_helper=sudo nova-rootwrap /etc/nova/rootwrap.conf
verbose=True
ec2_private_dns_show_ip=True
api_paste_config=/etc/nova/api-paste.ini
volumes_path=/var/lib/nova/volumes
enabled_apis=ec2,osapi_compute,metadata

auth_strategy = keystone
rpc_backend = rabbit
rabbit_host = 10.0.0.11
my_ip = 10.0.0.31
vnc_enabled = True
vncserver_listen = 0.0.0.0
vncserver_proxyclient_address = 10.0.0.31
novncproxy_base_url = http://192.168.100.11:6080/vnc_auto.html
glance_host = 10.0.0.11
vif_plugging_is_fatal=false
vif_plugging_timeout=0


[database]
connection = mysql://nova:NOVA_DBPASS@10.0.0.11/nova

[keystone_authtoken]
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = nova
admin_password = service_pass

删除sqlite

rm /var/lib/nova/nova.sqlite

重启compute服务

service nova-compute restart

编辑 /etc/sysctl.conf

net.ipv4.ip_forward=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0

马上生效

sysctl -p

安装网络组件

apt-get install -y neutron-common neutron-plugin-ml2 neutron-plugin-openvswitch-agent

编辑 /etc/neutron/neutron.conf

#core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
core_plugin = ml2

# service_plugins =
# Example: service_plugins = router,firewall,lbaas,vpnaas,metering
service_plugins = router

auth_strategy = keystone

allow_overlapping_ips = True

rpc_backend = neutron.openstack.common.rpc.impl_kombu

rabbit_host = 10.0.0.11

[keystone_authtoken]
#auth_host = 127.0.0.1
#auth_port = 35357
#auth_protocol = http
#admin_tenant_name = %SERVICE_TENANT_NAME%
#admin_user = %SERVICE_USER%
#admin_password = %SERVICE_PASSWORD%
#signing_dir = $state_path/keystone-signing
auth_uri = http://10.0.0.11:5000
auth_host = 10.0.0.11
auth_port = 35357
auth_protocol = http
admin_tenant_name = service
admin_user = neutron
admin_password = service_pass

编辑 /etc/neutron/plugins/ml2/ml2_conf.ini

[ml2]
type_drivers = gre
tenant_network_types = gre
mechanism_drivers = openvswitch

[ml2_type_gre]
tunnel_id_ranges = 1:1000

[ovs]
local_ip = 10.0.1.31
tunnel_type = gre
enable_tunneling = True

[securitygroup]
firewall_driver = neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
enable_security_group = True

重启OVS

service openvswitch-switch restart

再编辑 /etc/nova/nova.conf ，在[DEFAULT]里添加下面

network_api_class = nova.network.neutronv2.api.API
neutron_url = http://10.0.0.11:9696
neutron_auth_strategy = keystone
neutron_admin_tenant_name = service
neutron_admin_username = neutron
neutron_admin_password = service_pass
neutron_admin_auth_url = http://10.0.0.11:35357/v2.0
linuxnet_interface_driver = nova.network.linux_net.LinuxOVSInterfaceDriver
firewall_driver = nova.virt.firewall.NoopFirewallDriver
security_group_api = neutron

编辑 /etc/nova/nova-compute.conf ,修改为使用qemu

[DEFAULT]
compute_driver=libvirt.LibvirtDriver
[libvirt]
virt_type=qemu

重启相关服务

service nova-compute restart
service neutron-plugin-openvswitch-agent restart

安装就全部完成。

登录控制节点

root@controller:~# source creds 
root@controller:~# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-cert        controller                           internal         enabled       2014-09-02 10:31:03
nova-conductor   controller                           internal         enabled       2014-09-02 10:31:04
nova-scheduler   controller                           internal         enabled       2014-09-02 10:30:58
nova-consoleauth controller                           internal         enabled       2014-09-02 10:31:00
nova-compute     compute1                             nova             enabled       2014-09-02 10:30:57
root@controller:~#

1、命令行创建虚拟机

在控制节点上，运行下面的命令就可以。镜像我上面已经上传。下面的操作，你完全可以在Dashboard里进行操作，这里命令行下，了解更加深入。

下面的操作，在控制节点完成。

创建外部网络

source creds

#Create the external network:
neutron net-create ext-net --shared --router:external=True

#Create the subnet for the external network:
neutron subnet-create ext-net --name ext-subnet \
--allocation-pool start=192.168.100.101,end=192.168.100.200 \
--disable-dhcp --gateway 192.168.100.1 192.168.100.0/24

给租户创建内部网络

#Create the internal network:
neutron net-create int-net

#Create the subnet for the internal network:
neutron subnet-create int-net --name int-subnet \
--dns-nameserver 114.114.114.114 --gateway 172.16.1.1 172.16.1.0/24

创建路由，并且连接到外部网络

#Create the router:
neutron router-create router1

#Attach the router to the internal subnet:
neutron router-interface-add router1 int-subnet

#Attach the router to the external network by setting it as the gateway:
neutron router-gateway-set router1 ext-net

创建密钥

ssh-keygen

添加公钥

nova keypair-add --pub-key ~/.ssh/id_rsa.pub key1

设置安全组

# Permit ICMP (ping):
nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0

# Permit secure shell (SSH) access:
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0

创建虚拟机

NET_ID=$(neutron net-list | awk '/ int-net / { print $2 }')
nova boot --flavor m1.tiny --image cirros-0.3.2-x86_64 --nic net-id=$NET_ID \
--security-group default --key-name key1 instance1

查看虚拟机

nova list

申请公网IP

neutron floatingip-create ext-net

关联floating IP

nova floating-ip-associate instance1 192.168.100.102

这个时候，你会发现你在控制节点上，根本是无法访问 router 192.168.100.101和floating ip 192.168.100.102。

访问虚拟机，你需要登录网络节点上，你可以用下面命令访问虚拟机

# ip netns
qdhcp-bf7f3043-d696-4735-9bc7-8c2e4d95c8d5
qrouter-7e8bbb53-1ea6-4763-a69c-a0c875b5224b

第一个的虚拟机，第二个是路由器

# ip netns exec qdhcp-bf7f3043-d696-4735-9bc7-8c2e4d95c8d5 ifconfig
lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:16 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1216 (1.2 KB)  TX bytes:1216 (1.2 KB)

tap1a85db16-da Link encap:Ethernet  HWaddr fa:16:3e:ce:e0:e2  
          inet addr:172.16.1.3  Bcast:172.16.1.255  Mask:255.255.255.0
          inet6 addr: fe80::f816:3eff:fece:e0e2/64 Scope:Link
          UP BROADCAST RUNNING  MTU:1500  Metric:1
          RX packets:415 errors:0 dropped:0 overruns:0 frame:0
          TX packets:105 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:64724 (64.7 KB)  TX bytes:10228 (10.2 KB)

2、访问公网

你可能发现一个很明显的问题，你在网络节点是可以ping 通虚拟机的floating IP，router的IP，不过你在控制节点是无法访问的。

如果希望比较完美，实现虚拟机可以ping通公网，那么需要我们多了解一下内容才行。可以发现全部的流量都是通过192.168.100.21这个端口出去，我们需要设置一下这个端口，运行所有的IP和mac地址通过。

登录网络节点，通过ping 192.168.100.101 和192.168.100.102 ，获得他们的mac地址。

# arp -a
? (10.0.0.11) at fa:16:3e:34:d0:7a [ether] on eth0
? (192.168.100.102) at fa:16:3e:0c:be:cd [ether] on br-ex
? (10.0.1.31) at fa:16:3e:eb:96:1c [ether] on eth1
? (192.168.100.101) at fa:16:3e:0c:be:cd [ether] on br-ex
? (192.168.100.1) at fa:16:3e:c2:a8:a8 [ether] on br-ex

下面的操作，你可以在控制节点完成

通过curl获取token

使用token，修改192.168.100.21 port 的allow_address_pairs ，可以顺便把eth2和br-ex也修改，这样就不担心重启服务。

详细的操作，就参考这篇文档就可以。

http://www.chenshake.com/use-the-uos-api/

3、vnc访问

如果你登录Horizon，访问虚拟机，vnc可能无法访问，你需要登录uos，修改安全组规则。默认第一个虚拟机使用vnc的端口是6080。或者你全部打开端口。

参考资料

http://oddbit.com/rdo-hangout-multinode-packstack-slides/#/

https://github.com/ChaimaGhribi/OpenStack-Icehouse-Installation/blob/master/OpenStack-Icehouse-Installation.rst

参考文档 http://blog.oddbit.com/2014/05/23/open-vswitch-and-persistent-ma/

ovs-vsctl操作

root@network:~# ovs-vsctl show
05dd-bd0d-4af1-a331-c9394fbcb775
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.0.2"
root@network:~# ovs-vsctl add-br br-ex
root@network:~# ovs-vsctl show        
05dd-bd0d-4af1-a331-c9394fbcb775
    Bridge br-ex
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.0.2"
root@network:~# ovs-vsctl add-port br-ex eth2
root@network:~# ovs-vsctl show
05dd-bd0d-4af1-a331-c9394fbcb775
    Bridge br-ex
        Port "eth2"
            Interface "eth2"
        Port br-ex
            Interface br-ex
                type: internal
    Bridge br-int
        fail_mode: secure
        Port br-int
            Interface br-int
                type: internal
    ovs_version: "2.0.2"

网络节点重启服务

service neutron-plugin-openvswitch-agent restart
service neutron-dhcp-agent restart
service neutron-l3-agent restart
service neutron-metadata-agent restart
service dnsmasq restart

附录

建议采用vpn的方式，我就简单，采用pptp

PPTP，默认拨号连接上去，你本地就无法上网了。你需要设置一下。

去掉这个勾就可以了。

这个时候，你拨号连接上去，访问虚拟机的时候，走vpn通道，访问外网，还是走你以前的网络。

你还需要在你的增加一条路由才行，不过在win7，win8，增加路由，需要管理员权限。

按 Windows key + X, 再按 A

这时候，你就调出管理员权限的cmd

添加一条路由

route add 192.168.100.0 mask 255.255.255.0 10.100.100.1

这时候，你就可以ping通那台控制节点的虚拟机。不容易吧。

其他参考：http://www.tuicool.com/articles/jIVR3eM
http://www.aboutyun.com/thread-9670-1-1.html

乌托邦2号

关注

1
点赞
踩
3

收藏

觉得还不错? 一键收藏
0
评论
Openstack安装Openstack（Juno版本）

国内现在做Openstack的培训也开始多起来，不过应该都是采用虚拟机的方式来培训，我个人感觉还是很土的，不能吃自己的狗食。建议日后大家参加Openstack的培训的时候，都问一句，是不是在Openstack下培训Openstack。上次写过一篇Icehouse版本的。这次Juno版本，基本是一样的过程。不过调整一下顺序。用用户更容易操作。这次我是使用刻通云平台进行，希望可以做到更加流畅。
复制链接

扫一扫

专栏目录