使用Chef部署OpenStack (by quqi99)

作者：张华  发表于：2014-03-14
版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明

(http://blog.csdn.net/quqi99 )

Chef是一个类似于Puppet的用来快速部署软件及其依赖包的脚本工具,　将安装步骤通过脚本写出来（Puppet用基于XML的专用文法书写，Chef用Ruby书写），客户端从服务器端获取脚本并执行。其架构如下：

Chef是这样工作的:

• 在Workstation上定义各个Client应该如何配置自己(即Recipe）,然后将这些信息上传到中心服务器

• 每个Client连到中心服务器查看如何配置自己,然后进行自我配置

• Workstation与Server之间以及Client与Server之间通过pem作为认证，当新加一个Client的时候,需要从中心服务器上拷贝validator.pem到新加的Client，然后利用这个pem进行注册得到自己的client.pem进行以后的认证

1. 1. 1. 概念：Resource和Provider

Resource是Chef提供给你的用来描述系统的某一部分希望怎么配置(处于什么状态),请看例子:

package "vim" do
action :install
end
这就是一条Resource,它想要表达的是希望vim安装(处于安装的状态)

• 它有一个Resource类型(package)

• 有一个名字(vim)

• 可能还会有一些可选的参数(这个例子里没有)

• 有一个动作(install)(实际上描述一种状态,和Puppet里的ensure类似,不会每次都去install)

这里package是一个Resource类型,这里列出几个比较常用的Resource:

• Directory

• Execute

execute "ssh-keygen" do
command "ssh-keygen -t dsa -f /root/.ssh/id_rsa -N \"\""
if File.exists?("/root/.ssh/id_rsa")
action :nothing
end
end

• File

file "/tmp/something" do
owner "root" 
group "root" 
mode "0755" 
action :create
content "just test" 
end

• Group

# add group cyops and add root to it
group "cyops" do
system true
members "root"
end

• Package

• Script

• Service

service "ntpd" do
action[:enable,:start]
end

• Template, 下列把服务器上的config.conf.erb文件传到客户机上,重命名为config.conf并做变量替换(模板文件中的变量写为：<%=@config_var %>)

template "/tmp/config.conf" do
source "config.conf.erb" 
variables(
:config_var => node[:configs][:config_var]
)
end
找模板文件的顺序为：
.../template/host-client1.chefdemo.com/config.conf.erb
.../template/centos-6.5/config.conf.erb
.../template/centos/config.conf.erb
.../template/default/config.conf.erbUser 

user "random" do
comment "Random User"
uid 1000
gid "users"
home "/home/random"
shell "/bin/zsh"
action “create” # create是默认动作，故可以省略这一行
end

Provider的概念可能比较抽象,像上面的Resource的例子,我们之所以不关关心vim怎么被安装(apt,yum…),就是因为有Provider也就是说Provider负责把抽象的Resource对应到实际的命令(如上面的例子可能是:yum-y install vim)

1. 1. 1. 概念：Recipe

简单的说把多个Resource写到一起就是Recipe,客户端会把Recipe里面的Resouce按照顺序(重要)一条一条的应用到自身：

• 是Resource的组合

• 按顺序应用

• 可以包含其它的Recipe，例：include_recipe"ntp::default"

1. 1. 1. 概念：Node和Role

Role可以用来描述一台服务器希望被配置成什么样子(配置成web服务器,mysql服务器,甚至是一个论坛)

它有一个run_list,里面包含了要把一台服务器配置成这个样子所需要的Recipe和Role(Role可以包含Role)

Node很好理解,每一个被Chef管理的服务器(运行chef-client)就是一个Node

这里举一个例子帮助理解,有两个Recipe:ntp::default和mysql::default

package "ntp" do
action [:install]
end # 后面把这一条Resource简称为: 安装ntp的Resource
service "ntpd" do
action[:enable,:start]
end # 后面把这一条Resource简称为: 启动ntp的Resource 
package "mysql-server" do
action :install
end # 后面把这一条Resource简称为: 安装mysql-server的Resource 
service "mysql-server" do
action :start
end # 后面把这一条Resource简称为: 启动mysql-server的Resource 我们创建一个名叫ntp_and_mysql的Role并把这两个Recipe加到里面,相应的命令为

# knife role create ntp_and_mysql

这条命令会用vim打开一个文件让你编辑这个role,修改成这样然后保存退出,

{
"override_attributes": {
},
"chef_type": "role",
"env_run_lists": {
},
"json_class": "Chef::Role",
"name": "ntp_and_mysql",
"run_list": [
"recipe[ntp::default]",
"recipe[mysql::default]" 
],
"default_attributes": {
},
"description": "" 
}

然后把这个Role应用到一个Node上(实际上就是把这个Role的runlist里的Recipe加到Node的runlist里)

# knife node run list add client1.chefdemo.com 'role[ntp_and_mysql]'

最后client1.chefdemo.com这个Node会把它展开为4条Resource(按顺序)

安装ntp的Resource
启动ntp的Resource
安装mysql-server的Resource
启动mysql-server的Resource

再由Provider将其转为对应的命令,最后这个Node所要做的就是:

安装ntp
启动ntp
安装mysql-server
启动mysql-server

1. 1. 1. 概念：Cookbook

Cookbook实际上就是Recipe等一些东西的打包,像前面的ntp::default,ntp就是一个Cookbook

Cookbook的目录结构类似这样

tree /var/chef/cookbooks/ntp/
/var/chef/cookbooks/ntp/
├── attributes
├── definitions
├── files
│ └── default
├── libraries
├── metadata.rb
├── providers
├── README.md
├── recipes
│ ├── default.rb
│ └── ntp.rb
├── resources
└── templates
└── default
└── ntp.conf.erb
10 directories, 5 files

一个生成Cookbook目录结构的命令：rake new_cookbook COOKBOOK=test

1. 1. 1. 概念：DataBag

由于创建用户的那个Recipe就用到了DataBag,所以这里简单说一下

Data Bag提供了定义全局信息的方法,直接看例子

首先我们创建一个Data Bag

# knife data bag create admin

这条命令在chef-server上创建一个DataBag,可以在里面存储信息

mkdir -p /var/chef/data_bags/admin
vim /var/chef/data-bags/admin/quqi.json
{
"id": "quqi",
"shell": "/bin/bash",
"comment": "quqi",
"action": "create",

然后上传到服务端:

cd /var/chef

knife data bag from file admin quqi.json

现在就可以在Recipe里访问这些信息,可以有两个方法:data_bag和data_bag_item

• data_bag

admin用户下有quqi.json这一个数据文件那就data_bag('admin')就等于[“quqi”]

• data_bag_item

data_bag_item('admins', 'charlie')# => {"id"=>"william", "shell"=>"/bin/bash", "comment"=>"william", "action"=>"create"} 

概念：Attribute

属性(Attributes)就是节点(Node)的信息,如IP地址,主机名,加载的内核模块,系统中可用的编程语言的版本以及更多.新的属性可以用多种方式加到节点上.

有四种类型的属性,按优先级从高到低的顺序排列,它们是:

• automatic

• override

• normal

• default

Cookbook的属性文件可以在cookbook的attributes子目录中找到.他们在Node对象的上下文中运算并且使用Node的方法设置属性的值:

default["apache"]["dir"]          = "/etc/apache2"

这里Node对象的使用是隐含的,下面这样写和上面等价:

node.default["apache"]["dir"]          = "/etc/apache2"

概念：LWRP

LWRP(LightweightResources and Providers),自定义Resource和Provider.

用chef安装OpenStack

1. 1. 1. 环境准备

两个虚机通过vlan4090相连

chef-server,chef-workstation: 9.110.51.92 root/passw0rd

chef-client:9.110.51.153

1, 设置yum源

[yumcom]

name=openstack linux yum repository

baseurl=  <>

gpgcheck=0

enabled=1

2,编译生成openvswitchrpm for rhel6.5

yuminstall kernel-headers kernel-devel gcc make python-developenssl-devel kernel-devel, graphviz kernel-debug-devel automakerpm-build redhat-rpm-config libtool git

cd/bak/tools && wgethttp://ftp.gnu.org/gnu/autoconf/autoconf-2.64.tar.gz

tarxvf autoconf-2.64.tar.gz

cdautoconf-2.64/

./configure&& make && make install

cd/bak/tools && git clone git://git.openvswitch.org/openvswitch

cdopenvswitch/

./boot.sh&&./configure && make dist

mkdir-p /root/rpmbuild/SOURCES

cp/bak/tools/openvswitch/openvswitch-2.1.90.tar.gz/root/rpmbuild/SOURCES/

rpmbuild-bb rhel/openvswitch.spec
rpmbuild -bbrhel/openvswitch-kmod-rhel6.spec

rpm--nodeps -ivh/root/rpmbuild/RPMS/x86_64/openvswitch-2.1.90-1.x86_64.rpm

上述方法会遇到一个bug,http://permalink.gmane.org/gmane.network.openvswitch.devel/19542

是因为rhel6.5中已经有一个openvswitch-kmod,所以我们使用--nodeps参数只安装openvswitch-2.1.90-1.x86_64.rpm

3,设置网络(每台物理机都做类似设置),都只有一个网卡eth0,并将其配置成br-phy

cat/etc/sysconfig/network-scripts/ifcfg-br-phy

DEVICE=br-phy

NM_CONTROLLED=no

ONBOOT=yes

DEVICETYPE=ovs

TYPE=OVSBridge

BOOTPROTO=static

IPADDR=9.110.51.92

GATEWAY=9.110.51.1

BROADCAST=9.110.51.255

NETMASK=255.255.255.0

DNS1=9.0.148.50

IPV6INIT=no

cat/etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE=eth0

#HWADDR=52:54:00:8C:04:42

NM_CONTROLLED=no

ONBOOT=yes

DEVICETYPE=ovs

TYPE=OVSPort

IPV6INIT=no

OVS_BRIDGE=br-phy

 

chkconfigopenvswitch on

serviceopenvswitch start

servicenetwork restart

1. 给gem设置国内淘宝源避免伟大的长城防火墙的无端干扰,

   /opt/chef/embedded/bin/gemsources -a http://ruby.taobao.org/

1,Chef-server

rpm-Uvhhttps://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-server-11.0.10-1.el6.x86_64.rpm

cat/etc/hosts

9.110.51.92 chef-master

chef-server-ctl reconfigure  #必须在这一句之前配置hosts文件
# chef-server-ctl status

run: bookshelf: (pid 5587) 80883s; run: log: (pid 30526) 85923s

run: chef-expander: (pid 5606) 80882s; run: log: (pid 30487) 85924s

run: chef-server-webui: (pid 5610) 80882s; run: log: (pid 30668) 85906s

run: chef-solr: (pid 5621) 80881s; run: log: (pid 30443) 85930s

run: erchef: (pid 6374) 80774s; run: log: (pid 30562) 85917s

run: nginx: (pid 6361) 80774s; run: log: (pid 30834) 85895s

run: postgresql: (pid 5724) 80873s; run: log: (pid 30357) 85936s

run: rabbitmq: (pid 5728) 80872s; run: log: (pid 30083) 85953s

登录https://9.110.51.92 (admin/p@ssw0rd1)验证,登录不了,可能是因为没有配置hosts导致nginx没有正常启动

1. 1. 1. 2,Chef-client

rpm-Uvhhttps://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.10.4-1.el6.x86_64.rpm

3,Chef-workstation

1)安装配置chef-workstation

rpm-Uvhhttps://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.10.4-1.el6.x86_64.rpm

mkdir~/.chef

scp<chef-server-ip>:/etc/chef-server/*.pem ~/.chef/

mv~/.chef/chef-validation.pem ~/.chef/validation.pem#改名和下面运行'knifeconfigure –initial'命令输入的一致.

chmod600 ~/.chef/*.pem

# knife configure --initial
WARNING: No knife configuration file found
Where should I put the config file? [/root/.chef/knife.rb] 
Please enter the chef server URL: [http://chef-master:4000] https://9.110.51.92
Please enter a name for the new user: [root] 
Please enter the existing admin name: [admin] 
Please enter the location of the existing admin's private key: [/etc/chef/admin.pem] /root/.chef/admin.pem
Please enter the validation clientname: [chef-validator] 
Please enter the location of the validation key: [/etc/chef/validation.pem] /root/.chef/validation.pem
Please enter the path to a chef repository (or leave blank): 
Creating initial API user...
Please enter a password for the new user: 
[p@ssw0rd1]
Created user[root]
Configuration file written to /root/.chef/knife.rb

2) 上传 openstack cookbook

cd/bak/cookbook

gitclonehttps://github.com/stackforge/cookbook-openstack-compute.gitopenstack-compute

gitclonehttps://github.com/stackforge/cookbook-openstack-image.gitopenstack-image

gitclonehttps://github.com/stackforge/cookbook-openstack-identity.gitopenstack-identity

gitclonehttps://github.com/stackforge/cookbook-openstack-network.gitopenstack-network

gitclonehttps://github.com/stackforge/cookbook-openstack-dashboard.gitopenstack-dashboard

gitclonehttps://github.com/stackforge/cookbook-openstack-object-storage.gitopenstack-object-storage

gitclonehttps://github.com/stackforge/cookbook-openstack-block-storage.gitopenstack-block-storage

gitclonehttps://github.com/stackforge/cookbook-openstack-telemetry.gitopenstack-telemetry

gitclonehttps://github.com/stackforge/cookbook-openstack-orchestration.gitopenstack-orchestration

gitclonehttps://github.com/stackforge/cookbook-openstack-common.gitopenstack-common

gitclonehttps://github.com/stackforge/cookbook-openstack-ops-database.gitopenstack-ops-database

gitclonehttps://github.com/stackforge/cookbook-openstack-ops-messaging.gitopenstack-ops-messaging

gitclonehttps://github.com/opscode-cookbooks/apt.git

gitclonehttps://github.com/opscode-cookbooks/selinux.git

gitclonehttps://github.com/poise/python.git

gitclonehttps://github.com/opscode-cookbooks/database.git

gitclonehttps://github.com/opscode-cookbooks/mysql.git

gitclonehttps://github.com/hw-cookbooks/postgresql.git

gitclonehttps://github.com/opscode-cookbooks/aws.git

gitclonehttps://github.com/opscode-cookbooks/xfs.git

gitclonehttps://github.com/opscode-cookbooks/openssl.git

gitclonehttps://github.com/opscode-cookbooks/homebrew.git

gitclonehttps://github.com/opscode-cookbooks/windows.git

gitclonehttps://github.com/opscode-cookbooks/yum.git

gitclonehttps://github.com/opscode-cookbooks/yum-epel.git

gitclonehttps://github.com/opscode-cookbooks/apache2.git

gitclonehttps://github.com/opscode-cookbooks/iptables.git

gitclonehttps://github.com/opscode-cookbooks/logrotate.git

gitclonehttps://github.com/opscode-cookbooks/pacman.git

gitclonehttps://github.com/opscode-cookbooks/memcached.git

gitclonehttps://github.com/opscode-cookbooks/runit.git

gitclonehttps://github.com/opscode-cookbooks/rabbitmq.git

gitclonehttps://github.com/opscode-cookbooks/erlang.git

gitclonehttps://github.com/opscode-cookbooks/yum-erlang_solutions.git

gitclonehttps://github.com/opscode-cookbooks/chef_handler.git

gitclonehttps://github.com/etsy/statsd.git

配置cookbook的路径,在echo'cookbook_path ["/bak/cookbook"]' >> ~/.chef/knife.rb

上传cookbook至chef-server,注意:git clone的目录去掉cookbook-前缀

knifecookbook upload --all

在修改了cookbook之后,记得执行命令重新上传.

1. 上传role

gitclonehttps://github.com/stackforge/openstack-chef-repo.git

cdopenstack-chef-repo &&kniferole from file roles/*.rb && knife role-list

1. 上传Environment,并覆盖相关Attributes,属性都位于每个工程的attributes目录,假如,有一个属性为”default['openstack']['developer_mode']= True”, 那么在Environmnet中就可以这样写:

override_attributes(

"openstack"=> {

"developer_mode"=> true,

},

)

所以我们定义一个Environment如下,记得将developer_mode设置成true,避免给数据文件加密之类的操作:

cd/bak/cookbook/openstack-chef-repo && catenvironments/example.json (注意:此Environment我并没有测试)

{

"name":"openstack-test",

"description":"1 controller/n computes openstack deployment, using neutron(with vxlan tunnels between hosts) for the networking component.",

"cookbook_versions":{

},

"json_class":"Chef::Environment",

"chef_type":"environment",

"default_attributes":{

},

"override_attributes":{

"mysql":{

"allow_remote_root":true,

"root_network_acl":"%"

},

"openstack":{

"developer_mode":true,

"yum":{

"uri":"<your-yum-repository>",

"repo-key":"0",

"rdo_enabled":false

},

"endpoints":{

"host":"9.110.51.153"

},

"developer_mode":true,

"mq":{

"host":"9.110.51.153",

"bind_interface":"br-phy"

},

"db":{

"host":"9.110.51.153",

"bind_interface":"br-phy"

},

"auth":{

"validate_certs":false

},

"network":{

"debug":"True",

"use_namespaces":true,

"dhcp":{

"enable_isolated_metadata":"True"

},

"metadata":{

"nova_metadata_ip":"9.110.51.153"

},

"openvswitch":{

"tunnel_id_ranges":"1:1000",

"enable_tunneling":"True",

"tenant_network_type":"vxlan",

"local_ip_interface":"br-phy"

},

"api":{

"bind_interface":"br-phy"

}

},

"image":{

"api":{

"bind_interface":"br-phy"

},

"registry":{

"bind_interface":"br-phy"

},

"image_upload":true,

"upload_images":[

"cirros"

],

"upload_image":{

"cirros":"https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img"

}

},

"compute":{

"xvpvnc_proxy":{

"bind_interface":"br-phy"

},

"novnc_proxy":{

"bind_interface":"br-phy"

},

"libvirt":{

"virt_type":"qemu"

},

"network":{

"public_interface":"br-phy",

"service_type":"neutron"

},

"config":{

"ram_allocation_ratio":5.0

}

}

}

}

}

上传Envvironments,cd/bak/cookbook/openstack-chef-repo &&

knifeenvironment from file environments/example.json

1. 部署服务到chef-client,执行下列命令时,记得在所有的chef-client配置chef-server的hosts文件,这里是(9.110.51.92chef-master):

   cp-r bootstrap ~/.chef && cd ~

knifebootstrap 9.110.51.153 --ssh-user root --ssh-password passw0rd -Eopenstack-test --run-list role["os-ops-messaging"]

knifebootstrap 9.110.51.153 --ssh-user root --ssh-password passw0rd -Eopenstack-test --run-list "role["os-identity"]"

knifebootstrap 9.110.51.153 --ssh-user root --ssh-password passw0rd -Eopenstack-test --run-list "role["os-image"]"

knifebootstrap 9.110.51.153 --ssh-user root --ssh-password passw0rd -Eopenstack-test --run-list "role["os-network"]"

knifebootstrap9.110.51.153--ssh-user root --ssh-password passw0rd -E openstack-test--run-list "role["os-compute-setup"]"

knifebootstrap9.110.51.153--ssh-user root --ssh-password passw0rd-E openstack-test --run-list "role[os-compute-conductor]"

knifebootstrap9.110.51.153--ssh-user root --ssh-password passw0rd -E openstack-test --run-list "role[os-compute-scheduler]"

knifebootstrap9.110.51.153--ssh-user root --ssh-password passw0rd-E openstack-test --run-list "role[os-compute-api]"

knifebootstrap9.110.51.153--ssh-user root --ssh-password passw0rd-Eopenstack-test --run-list "role[os-compute-cert]"

knifebootstrap9.110.51.153--ssh-user root --ssh-password passw0rd-E openstack-test --run-list "role[os-compute-vncproxy]"

knifebootstrap9.110.51.153--ssh-user root --ssh-password passw0rd-E openstack-test --run-list"recipe[openstack-compute::compute]"

#下面的allinone-compute将安装all-in-one的环境

# knifebootstrap 9.110.51.153--ssh-userroot --ssh-password passw0rd -E openstack-test--run-listrole[allinone-compute]

1. 1. 1. 参考：

http://williamherry.com/blog/2012/07/16/chef-basic/

http://williamherry.com/blog/2012/08/31/chef-tips/
http://www.server110.com/openstack/201310/2939.html 
RubyCc.com is for sale | HugeDomains 
http://xinkang120.blog.163.com/blog/static/194668223201232731237547/ 
http://heylinux.com/archives/2208.html 
Chef/GettingStarted - OpenStack 
gettingstartedwithchef.com - 
http://developer.rackspace.com/blog/understanding-the-chef-environment-file-in-rackspace-private-cloud.html
openstack/cookbook-openstack-network - Sourcegraph

https://github.com/search?q=%40stackforge+cookbook

http://heylinux.com/archives/2208.html