使用Juju将OpenStack部署在单机的LXD容器上(by quqi99)

原创 2016年08月05日 19:54:29

**作者:张华 发表于:2016-08-05
版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明
http://blog.csdn.net/quqi99 )**

理论基础

  1. iscsi还不能运行在容器里(因为netlink还不支持namesapce),本文采用rbd使用ceph代替iscsi
  2. ovs, kvm通过定义profile支持运行在容器里。ovs目前只支持security.privileged: “true”

配置LXD

参考Play with LXD一文 在ubuntu 16.04上部署LXD环境。

LXD上部署OpenStack

1, 从这个链接下载 ‘openstack-base.zip’ ,里面有下面要用到的bundle.yaml
2, 运行’juju bootstrap’,注意:运行这一步时先不要修改profile

#sudo snap install lxd
export PATH=/snap/bin:$PATH
juju bootstrap --debug --config bootstrap-series=xenial --config agent-stream=devel localhost lxd-controller
lxc exec `lxc list |grep juju- |awk -F '|' '{print $2}'` bash

3, 创建model,且它会自动生成juju-openstack-model profile (’juju add-model’会自动执行这一句‘lxc profile create juju-openstack-model 2>/dev/null || echo “juju-openstack-model profile already exists”’), 如果不定义model,就会有一个名为default的model,那么这时下面第4步要编辑juju-default profile

juju add-model openstack-model
juju models
lxc profile show juju-openstack-model

4, 编辑juju-openstack-model profile。

sudo apt-get install --reinstall linux-image-extra-$(uname -r)
sudo modprobe nbd
sudo modprobe ip_tables
sudo modprobe openvswitch

cat << EOF > juju-openstack-model.yaml
name: juju-openstack-model
config:
  boot.autostart: "true"
  security.nesting: "true"
  security.privileged: "true"
  raw.lxc: lxc.aa_profile=unconfined
  linux.kernel_modules: openvswitch,nbd,ip_tables,ip6_tables,ebtables,netlink_diag,nf_nat,overlay
devices:
  eth0:
    mtu: "9000"
    name: eth0
    nictype: bridged
    parent: lxdbr1
    type: nic
  eth1:
    mtu: "9000"
    name: eth1
    nictype: bridged
    parent: lxdbr1
    type: nic
  kvm:
    path: /dev/kvm
    type: unix-char
  mem:
    path: /dev/mem
    type: unix-char
  root:
    path: /
    pool: default
    type: disk
  tun:
    path: /dev/net/tun
    type: unix-char
EOF
cat lxd-profile.yaml | lxc profile edit juju-openstack-model
#其他命令演示
#lxc profile set juju-openstack-model raw.lxc lxc.aa_profile=unconfined
#lxc profile device add juju-openstack-model fuse unix-char path=/dev/fuse
#/snap/bin/lxc network create lxdbr1 ipv4.address=auto ipv4.nat=true ipv6.address=none

5, 使用juju一键部署openstack

wget https://api.jujucharms.com/charmstore/v5/openstack-base/archive/bundle.yaml
juju deploy bundle.yaml
juju status
juju debug-log

安装过程中遇到的问题

  • 如果报这个错 - failed to bootstrap model: cannot start bootstrap instance: The container’s root device is missing the pool property, 那是要在profile中的root元素下添加:pool: default
  • bootstrap时报这个错 - FATAL: Module ip6_tables,ebtables,netlink_diag not found in directory /lib/modules/4.4.0-98-generic - 运行‘sudo apt-get install –reinstall linux-image-extra-(unamer)/lib/modules/(uname -r)/kernel/net/netlink/netlink_diag.ko)。此外是因为profile中写的这些模块名是从网页拷过来的存在乱码

配置使用OpenStack

source novarc
$ cat novarc 
#!/bin/bash
export OS_USERNAME=admin
export OS_PASSWORD=openstack
export OS_TENANT_NAME=admin
export OS_REGION_NAME=RegionOne
export OS_AUTH_URL=${OS_AUTH_PROTOCOL:-http}://`juju run --unit  keystone/0 "unit-get private-address"`:5000/v2.0

curl http://cloud-images.ubuntu.com/xenial/current/xenial-server-cloudimg-amd64-disk1.img
openstack image create --public --container-format=bare --disk-format=qcow2 xenial

./neutron-ext-net -g 10.0.8.1 -c 10.0.8.0/24 \ -f 10.0.8.201:10.0.8.254 ext_net
./neutron-tenant-net -t admin -r provider-router \ -N 10.0.8.1 internal 192.168.20.0/24

nova keypair-add --pub-key ~/.ssh/id_rsa.pub mykey
nova boot --image xenial --flavor m1.small --key-name mykey --nic net-id=$(neutron net-list | grep internal | awk '{ print $2 }') i1

cinder create --name testvolume 10
nova volume-attach xenial $(cinder list | grep testvolume | awk '{ print $2 }') /dev/vdc

nova floating-ip-create 
nova add-floating-ip <uuid-of-instance> <new-floating-ip>

neutron security-group-rule-create --protocol icmp --direction ingress $(nova secgroup-list | grep default | awk '{ print $2 }') 
neutron security-group-rule-create --protocol tcp  --port-range-min 22 --port-range-max 22  --direction ingress $(nova secgroup-list | grep default | awk '{ print $2 }')

ssh ubuntu@<new-floating-ip>

又一例 - 部署opencontrail在lxd单机上

下面的yaml是juju2.0的,如果是juju1.x可见:http://pastebin.ubuntu.com/24170320/
实际上,opencontrail vrouter部署在容器里会报下列错,此例子只是说明yaml怎么写。

2017-03-13 11:46:06 INFO juju-log Loading kernel module vrouter
2017-03-13 11:46:06 INFO install modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.8.0-34-generic/modules.dep.bin'
2017-03-13 11:46:06 INFO juju-log vrouter kernel module failed to load, clearing pagecache and retrying
series: trusty
services:
  # openstack
  ubuntu:
    charm: cs:trusty/ubuntu
    num_units: 1
  ntp:
    charm: cs:trusty/ntp
  mysql:
    charm: cs:trusty/mysql
    options:
      dataset-size: 15%
      max-connections: 1000
    num_units: 1
  rabbitmq-server:
    charm: cs:trusty/rabbitmq-server
    num_units: 1
  keystone:
    charm: cs:~sdn-charmers/trusty/keystone
    options:
      admin-password: password
      admin-role: admin
      openstack-origin: cloud:trusty-mitaka
    num_units: 1
  nova-cloud-controller:
    charm: cs:trusty/nova-cloud-controller
    options:
      network-manager: Neutron
      openstack-origin: cloud:trusty-mitaka
    num_units: 1
  neutron-api:
    charm: cs:trusty/neutron-api
    options:
      manage-neutron-plugin-legacy-mode: false
      openstack-origin: cloud:trusty-mitaka
    num_units: 1
  glance:
    charm: cs:trusty/glance
    options:
      openstack-origin: cloud:trusty-mitaka
    num_units: 1
  openstack-dashboard:
    charm: cs:trusty/openstack-dashboard
    options:
      openstack-origin: cloud:trusty-mitaka
    num_units: 1
  nova-compute:
    charm: cs:trusty/nova-compute
    options:
      openstack-origin: cloud:trusty-mitaka
    num_units: 1
  # contrail
  cassandra:
    charm: cs:trusty/cassandra
    options:
      authenticator: AllowAllAuthenticator
      install_sources: |
        - deb http://www.apache.org/dist/cassandra/debian 22x main
        - ppa:openjdk-r/ppa
        - ppa:stub/cassandra
    num_units: 1
  zookeeper:
    charm: cs:~charmers/trusty/zookeeper
    num_units: 1
  kafka:
    charm: cs:~sdn-charmers/trusty/apache-kafka
    num_units: 1
  contrail-configuration:
    charm: cs:~sdn-charmers/trusty/contrail-configuration
    options:
      openstack-origin: cloud:trusty-mitaka
    num_units: 1
  contrail-control:
    charm: cs:~sdn-charmers/trusty/contrail-control
    num_units: 1
  contrail-analytics:
    charm: cs:~sdn-charmers/trusty/contrail-analytics
    num_units: 1
  contrail-webui:
    charm: cs:~sdn-charmers/trusty/contrail-webui
    num_units: 1
  neutron-api-contrail:
    charm: cs:~sdn-charmers/trusty/neutron-api-contrail
    num_units: 0
  neutron-contrail:
    charm: cs:~sdn-charmers/trusty/neutron-contrail
    num_units: 0

relations:
  # openstack
 - [ ubuntu, ntp ]
 - [ keystone, mysql ]
 - [ glance, mysql ]
 - [ glance, keystone ]
 - [ nova-cloud-controller, mysql ]
 - [ nova-cloud-controller, rabbitmq-server ]
 - [ nova-cloud-controller, keystone ]
 - [ nova-cloud-controller, glance ]
 - [ neutron-api, mysql ]
 - [ neutron-api, rabbitmq-server ]
 - [ neutron-api, nova-cloud-controller ]
 - [ neutron-api, keystone ]
 - [ neutron-api, neutron-api-contrail ]
 - [ "nova-compute:shared-db", "mysql:shared-db" ]
 - [ "nova-compute:amqp", "rabbitmq-server:amqp" ]
 - [ nova-compute, glance ]
 - [ nova-compute, nova-cloud-controller ]
 - [ nova-compute, ntp ]
 - [ openstack-dashboard, keystone ]
  # contrail
 - [ kafka, zookeeper ]
 - [ "contrail-configuration:cassandra", "cassandra:database" ]
 - [ contrail-configuration, zookeeper ]
 - [ contrail-configuration, rabbitmq-server ]
 - [ "contrail-configuration:identity-admin", "keystone:identity-admin" ]
 - [ "contrail-configuration:identity-service", "keystone:identity-service" ]
 - [ neutron-api-contrail, contrail-configuration ]
 - [ neutron-api-contrail, keystone ]
 - [ "contrail-control:contrail-api", "contrail-configuration:contrail-api" ]
 - [ "contrail-control:contrail-discovery", "contrail-configuration:contrail-discovery" ]
 - [ "contrail-control:contrail-ifmap", "contrail-configuration:contrail-ifmap" ]
 - [ contrail-control, keystone ]
 - [ "contrail-analytics:cassandra", "cassandra:database" ]
 - [ contrail-analytics, kafka ]
 - [ contrail-analytics, zookeeper ]
 - [ "contrail-analytics:contrail-api", "contrail-configuration:contrail-api" ]
 - [ "contrail-analytics:contrail-discovery", "contrail-configuration:contrail-discovery" ]
 - [ "contrail-analytics:identity-admin", "keystone:identity-admin" ]
 - [ "contrail-analytics:identity-service", "keystone:identity-service" ]
 - [ "contrail-configuration:contrail-analytics-api", "contrail-analytics:contrail-analytics-api" ]
 - [ nova-compute, neutron-contrail ]
 - [ "neutron-contrail:contrail-discovery", "contrail-configuration:contrail-discovery" ]
 - [ "neutron-contrail:contrail-api", "contrail-configuration:contrail-api" ]
 - [ neutron-contrail, keystone ]
 - [ contrail-webui, keystone ]
 - [ "contrail-webui:cassandra", "cassandra:database" ]

通过conjure-up安装OpenStack

我们也可以通过conjure-up安装OpenStack,

#Install a lxd container
sudo lxc init ubuntu:16.04 openstack -c security.privileged=true -c security.nesting=true -c "linux.kernel_modules=iptable_nat, ip6table_nat, ebtables, openvswitch, nbd"
printf "lxc.cap.drop=\nlxc.aa_profile=unconfined\n" | sudo lxc config set openstack raw.lxc -
sudo lxc config get openstack raw.lxc
lxc config device add openstack mem unix-char path=/dev/mem
lxc start openstack
lxc list

#Install conjure-up inside the lxd container
#lxc exec openstack bash
lxc exec openstack -- apt update
#lxc exec openstack -- apt dist-upgrade -y
lxc exec openstack -- apt install squashfuse -y
lxc exec openstack -- ln -s /bin/true /usr/local/bin/udevadm
lxc exec openstack -- snap install conjure-up --classic

#Init lxd container
#Use the “dir” storage backend (“zfs” doesn’t work in a nested container)
#Do NOT configure IPv6 networking (conjure-up/juju don’t play well with it)
#lxc exec openstack -- lxd init
lxc exec openstack -- snap install lxd
sleep 10  #avoid the error 'Unable to talk to LXD: Get http://unix.socket/1.0'
lxc exec openstack -- /snap/bin/lxd init --auto
lxc exec openstack -- /snap/bin/lxc network create lxdbr0 ipv4.address=auto ipv4.nat=true ipv6.address=none
lxc exec openstack -- /snap/bin/lxc profile show default

#Deploying OpenStack with conjure-up in nested LXD
#conjure-up is a nice, user friendly, tool that interfaces with Juju to deploy complex services.
#Step 1, select “OpenStack with NovaLXD”
#Step 2, select “localhost” as the deployment target (uses LXD)
#Step 3, select default in all middle steps, and click “Deploy all remaining applications”
lxc exec openstack -- sudo -u ubuntu -i conjure-up
hua@node1:~$ sudo lxc list
+-----------+---------+--------------------------------+------+------------+-----------+
|   NAME    |  STATE  |              IPV4              | IPV6 |    TYPE    | SNAPSHOTS |
+-----------+---------+--------------------------------+------+------------+-----------+
| openstack | RUNNING | 10.73.227.154 (eth0)           |      | PERSISTENT | 0         |
|           |         | 10.164.92.1 (lxdbr0)           |      |            |           |
|           |         | 10.101.0.1 (conjureup0)        |      |            |           |
+-----------+---------+--------------------------------+------+------------+-----------+

#Or deploy OpenStack with conjure-up in physical node
sudo snap install lxd
export PATH=/snap/bin:$PATH
sudo /snap/bin/lxd init --auto
sudo /snap/bin/lxc network create lxdbr0 ipv4.address=auto ipv4.nat=true ipv6.address=none
sudo -i
conjure-up openstack #but I hit the error 'This should _not_ be run as root or with sudo' even though I've already used root

下面粘一些使用conjure-up过程中的截图:
这里写图片描述
这里写图片描述
这里写图片描述

参考

版权声明:本文为博主原创文章,如需转载,请注明出处!

将kubernetes跑在本地LXD容器中(by quqi99)

版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明 (http://blog.csdn.net/quqi99)问题本文将kubernetest跑在本地LXD容器中。...
  • quqi99
  • quqi99
  • 2017年11月12日 21:18
  • 272

存储之二 深入理解lun,zone的概念

SAN是满足迅速增长的企业存储需求的最具发展前景的手段,而存储交换机/路由器则使构建和管理SAN变得更容易。随着SAN交换设备出现在存储环境中,IT人员可以利用现有的知识方便地构建存储网络。    ...

Linux上每个SCSI设备的最大LUN数目是多少(by quqi99)

作者:张华  发表于:2016-04-28版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明( http://blog.csdn.net/quqi99 )SCSI、...
  • quqi99
  • quqi99
  • 2016年04月28日 18:10
  • 7229

OpenStack中ipv6的设计与使用的一些理论分析(未测试)( by quqi99 )

OpenStack中ipv6的设计与使用的一些理论分析(未测试)( by quqi99 ) 作者:张华  发表于:2013-03-29 版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始...
  • quqi99
  • quqi99
  • 2013年03月29日 17:11
  • 3491

为什么openstack中的oslo模块总喜欢发生代码冲突? (by quqi99)

作者:张华  发表于:2014-02-02版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明(http://blog.csdn.net/quqi99 )      ...
  • quqi99
  • quqi99
  • 2015年02月02日 14:50
  • 6016

Fedora 16上源码建立pydev + eclipse的OpenStack开发环境笔记草稿 ( by quqi99 )

Fedora 16上源码建立pydev + eclipse的OpenStack开发环境笔记草稿  ( by quqi99 ) 作者:张华  发表于:2012-3-30 版权声明:可以任意转载,...
  • quqi99
  • quqi99
  • 2012年03月30日 13:49
  • 9211

OpenStack Neutron FWaaS 学习 ( by quqi99 )

OpenStack Neutron FWaaS 学习 ( by quqi99 ) 作者:张华  发表于:2013-06-24 版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和...
  • starean
  • starean
  • 2013年06月27日 10:52
  • 984

Test OpenStack SRIOV (by quqi99)

**作者:张华 发表于:2016-12-06 版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明 ( http://blog.csdn.net/quqi99 ...
  • quqi99
  • quqi99
  • 2016年12月06日 16:10
  • 1338

OpenStack中遇到的MTU问题 ( by quqi99 )

OpenStack中遇到的MTU问题 ( by quqi99 ) 作者:张华  发表于:2013-11-10 版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明...
  • quqi99
  • quqi99
  • 2013年11月10日 17:31
  • 6072

调试OpenStack时遇到的主要问题(by quqi99)

作者:张华  发表于:2014-11-09版权声明:可以任意转载,转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明( http://blog.csdn.net/quqi99 )...
  • quqi99
  • quqi99
  • 2014年11月09日 15:08
  • 8043
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:使用Juju将OpenStack部署在单机的LXD容器上(by quqi99)
举报原因:
原因补充:

(最多只允许输入30个字)