版本说明:
CloudFoundry:V2版本-v149+(本文后续章节中的Manifest适用于149+版本的CF Release)
OpenStack:Grizzly版本+nova-network
CloudFoundry部署交流QQ群:176302388
CloudFoundry目前官方的版本为V2版,release最新发布为149,不过CloudFoundry在GitHub上的仓库代码贡献非常频繁,一般只要提出有BUG,很快就会有修改并提交,所以,最好的部署方式是直接下载源码,创建dev版的release进行部署,当然,也可以直接使用官方发布的release进行打包部署,本文中使用149.1-dev进行打包部署,如果要使用官方release,指定release路径即可,如:bosh create release release/cf-149.yml。
注意:本文中的示例Manifest仅适用于149+版本的release。
1、调整虚拟机存储配置
默认情况下blobstore/0虚拟机中/var/vcap/store的存储空间比较小,而后边上传用于部署CloudFoundry的Stemcell包和Release包时,会因为存储空间不足导致错误,所以这里先增加/var/vcap/store的挂载空间,防止发生错误。
具体步骤参看文章:http://blog.csdn.net/tiger435/article/details/8808623#t8。
说明:本环境配置文件中的所有通过Stemcell创建的虚拟机的登陆账户密码为“root/c1oudc0w”或者“vcap/c1oudc0w”
注意:即使配置挂载空间后,以后重启blobstore的虚拟机的时候一定记得再次挂载,或者可以修改为开机挂载,本人就是忘记该步骤,导致重启虚拟机后,后边部署cf-service-release时出现莫名其妙找不到package的错误,折磨几个小时。
2、下载并编译CloudFoundry发布包
依次执行以下命令从GitHub获取并更新代码
root@bosh-cli:~# mkdir -p ~/src/cloudfoundry
root@bosh-cli:~# cd ~/src/cloudfoundry
root@bosh-cli:~/src/cloudfoundry# git clone -b release-candidate git://github.com/cloudfoundry/cf-release.git
root@bosh-cli:~/src/cloudfoundry# cd cf-release
root@bosh-cli:~/src/cloudfoundry/cf-release# ./update
下载并更新完毕之后,执行以下命令创建release包,这是一个相当漫长的过程,中间还会发生多次中断,只要重新执行bosh create release --force即可:
说明:bosh create release --force 是使用最新的CloudFoundry源码创建release-dev包,可能会存在一些奇怪的BUG或者错误、问题,好处就是使用这种方式构建release-dev包,下次代码更新后,再进行打包速度会比较快。若过程中出现问题,可以使用经过测试发布的release包,打包方式:bosh create release releases/cf-149.yml,会在releases目录下生成一个tgz的压缩包,之后执行bosh upload releases/cf-149.tgz上传release包即可。
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh create release --force
....
Building jobs
-------------
Building health_manager_next...
Final version: NOT FOUND
Dev version: NOT FOUND
Generating...
Generated version 7.1-dev
Building dashboard...
Final version: Downloading `dashboard (10)'...
Building ccdb_postgres...
Final version: Downloading `ccdb_postgres (7)'...
Building gorouter...
Final version: Downloading `gorouter (2)'...
Building dea_next...
Final version: NOT FOUND
Dev version: NOT FOUND
Generating...
Generated version 12.1-dev
Building collector...
Final version: NOT FOUND
Dev version: NOT FOUND
Generating...
Generated version 6.1-dev
Building uaa...
Final version: NOT FOUND
Dev version: NOT FOUND
Generating...
Generated version 27.1-dev
Building syslog_aggregator...
Final version: NOT FOUND
Dev version: NOT FOUND
Generating...
Generated version 10.1-dev
Building login...
Final version: NOT FOUND
Dev version: NOT FOUND
Generating...
Generated version 11.1-dev
Building debian_nfs_server...
Final version: FOUND LOCAL
Building nats...
Final version: NOT FOUND
Dev version: NOT FOUND
Generating...
Generated version 10.1-dev
Building postgres...
Final version: Downloading `postgres (4)'...
Building cloud_controller_ng...
Final version: NOT FOUND
Dev version: NOT FOUND
Generating...
Generated version 7.1-dev
Building release
----------------
Generating manifest...
----------------------
Writing manifest...
Release summary
---------------
Packages
+---------------------+----------+-------------+
| Name | Version | Notes |
+---------------------+----------+-------------+
| git | 1 | |
| nginx | 9 | |
| ruby | 8 | |
| health_manager_next | 18 | |
| dashboard | 14 | |
| libyaml | 1 | |
| rootfs_lucid64 | 1 | |
| insight_agent | 2 | |
| ruby_next | 3 | |
| gorouter | 4.1-dev | new version |
| dea_next | 16.1-dev | new version |
| collector | 8.1-dev | new version |
| uaa | 24.1-dev | new version |
| golang | 2 | |
| warden | 26.1-dev | new version |
| mysqlclient | 3 | |
| syslog_aggregator | 4 | |
| libpq | 5 | |
| login | 13.1-dev | new version |
| dea_jvm | 4 | |
| buildpack_cache | 1.1-dev | new version |
| debian_nfs_server | 3 | |
| nats | 8 | |
| postgres | 4 | |
| sqlite | 3 | |
| imagemagick | 2 | |
| cloud_controller_ng | 13.1-dev | new version |
| common | 5 | |
+---------------------+----------+-------------+
Jobs
+---------------------+----------+-------------+
| Name | Version | Notes |
+---------------------+----------+-------------+
| health_manager_next | 7.1-dev | new version |
| dashboard | 10 | |
| ccdb_postgres | 7 | |
| gorouter | 2 | |
| dea_next | 12.1-dev | new version |
| collector | 6.1-dev | new version |
| uaa | 27.1-dev | new version |
| syslog_aggregator | 10.1-dev | new version |
| login | 11.1-dev | new version |
| debian_nfs_server | 7 | |
| nats | 10.1-dev | new version |
| postgres | 4 | |
| cloud_controller_ng | 7.1-dev | new version |
+---------------------+----------+-------------+
Jobs affected by changes in this release
+---------------------+----------+
| Name | Version |
+---------------------+----------+
| health_manager_next | 7.1-dev |
| dea_next | 12.1-dev |
| collector | 6.1-dev |
| uaa | 27.1-dev |
| syslog_aggregator | 10.1-dev |
| login | 11.1-dev |
| nats | 10.1-dev |
| cloud_controller_ng | 7.1-dev |
| gorouter | 2 |
+---------------------+----------+
Release version: 149.1-dev
Release manifest: /root/src/cloudfoundry/cf-release/dev_releases/cf-release-149.1-dev.yml
3、切换bosh target到Bosh的Director并创建新用户
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh target https://10.68.19.132:25555
Target set to `bosh'
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh login
Your username: admin
Enter password: *****
Logged in as `admin'
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh create user
Enter new username: venusource
Enter new password: **********
Verify new password: **********
User `venusource' has been created
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh login
Your username: venusource
Enter password: **********
Logged in as `venusource'
4、上传release和stemcell到Bosh中
上传Stemcell和部署Bosh的时候使用的是同一个Stemcell:
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh upload stemcell ~/bosh-workspace/stemcells/bosh-stemcell-latest-openstack-kvm-ubuntu.tgz
Verifying stemcell...
File exists and readable OK
Using cached manifest...
Stemcell properties OK
Stemcell info
-------------
Name: bosh-stemcell
Version: 776
Checking if stemcell already exists...
No
Uploading stemcell...
latest-bosh-s: 100% |oooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo| 251.3MB 11.0MB/s Time: 00:00:22
Director task 1
Update stemcell
extracting stemcell archive (00:00:12)
verifying stemcell manifest (00:00:00)
checking if this stemcell already exists (00:00:00)
uploading stemcell bosh-stemcell/776 to the cloud (00:02:05)
save stemcell bosh-stemcell/776 (4e433167-acf2-41b0-b691-934684a63e52) (00:00:00)
Done 5/5 00:02:17
Task 1 done
Started 2013-12-17 07:03:06 UTC
Finished 2013-12-17 07:05:23 UTC
Duration 00:02:17
Stemcell uploaded and created
上传Release到Bosh:
root@bosh-cli:~/src/cloudfoundry/cf-release# bosh upload release
....
Release has been created
cf-release/149.1-dev (00:00:00)
Done 1/1 00:00:00
Task 2 done
Started 2013-12-18 01:59:30 UTC
Finished 2013-12-18 02:02:20 UTC
Duration 00:02:50
Release uploaded
验证下Stemcell和Release是否正常:
root@bosh-cli:~# bosh releases
+------------+------------+-------------+
| Name | Versions | Commit Hash |
+------------+------------+-------------+
| cf-release | 149.1-dev* | 27784f08 |
+------------+------------+-------------+
(*) Currently deployed
(+) Uncommitted changes
Releases total: 3
root@bosh-cli:~# bosh stemcells
+---------------------------+---------+--------------------------------------+
| Name | Version | CID |
+---------------------------+---------+--------------------------------------+
| bosh-openstack-kvm-ubuntu | 1478 | 316501e1-482d-472d-9fd9-b71a7b1223ee |
+---------------------------+---------+--------------------------------------+
Stemcells total: 1
5、创建并配置用于部署CloudFoundry的Manifest文件
从GitHub中获取模板文件:root@bosh-cli:~/src/cloudfoundry/cf-release# cd ~/bosh-workspace/deployments/
root@bosh-cli:~/bosh-workspace/deployments# vi cf-149.yml
下边给我本环境中配置后的Manifest.yml文件作为参考:
---
name: cf
director_uuid: 785aa0a6-87c7-4452-a862-c118afc90b37
releases:
- name: cf-release
version: latest
networks:
- name: floating
type: vip
cloud_properties: {}
- name: default
type: dynamic
cloud_properties:
security_groups:
- default
update:
canaries: 1
canary_watch_time: 30000-60000
update_watch_time: 30000-60000
max_in_flight: 4
compilation:
workers: 6
network: default
reuse_compilation_vms: true
cloud_properties:
instance_type: v1.medium
resource_pools:
- name: small
network: default
size: 4
stemcell:
name: bosh-openstack-kvm-ubuntu
version: latest
cloud_properties:
instance_type: v1.small
- name: medium
network: default
size: 0
stemcell:
name: bosh-openstack-kvm-ubuntu
version: latest
cloud_properties:
instance_type: v1.medium
jobs:
- name: data
release: cf-release
template:
- postgres
- debian_nfs_server
instances: 1
resource_pool: small
persistent_disk: 4096
networks:
- name: default
default:
- dns
- gateway
properties:
db: databases
- name: core
release: cf-release
template:
- syslog_aggregator
- nats
- health_manager_next
- uaa
instances: 1
resource_pool: small
networks:
- name: default
default:
- dns
- gateway
- name: api
release: cf-release
template:
- cloud_controller_ng
- gorouter
instances: 1
resource_pool: small
networks:
- name: default
default:
- dns
- gateway
- name: floating
static_ips:
- 10.68.19.133
properties:
db: databases
- name: dea
release: cf-release
template:
- dea_next
instances: 1
resource_pool: small
networks:
- name: default
default: [dns, gateway]
properties:
cf:
name: cf
dns: vsc.com
ip_addresses: ["10.68.19.133"]
deployment_size: medium
security_group: default
persistent_disk: 4096
common_password: c1oudc0w
dea_server_ram: 4096
domain: vsc.com
system_domain: vsc.com.com
system_domain_organization: system_domain
app_domains:
- vsc.com
networks:
apps: default
management: default
nats:
address: 0.core.default.cf.bosh
machines:
- 0.core.default.cf.bosh
port: 4222
user: nats
password: c1oudc0w
authorization_timeout: 5
router:
address: 0.api.default.cf.bosh
port: 8081
status:
port: 8080
user: gorouter
password: c1oudc0w
dea: &dea
memory_mb: 4096
disk_mb: 10240
directory_server_protocol: http
dea_next: *dea
syslog_aggregator:
address: 0.core.default.cf.bosh
port: 54321
nfs_server:
address: 0.data.default.cf.bosh
network: "*.cf.bosh"
idmapd_domain: vsc.com
debian_nfs_server:
no_root_squash: true
databases: &databases
db_scheme: postgres
address: 0.data.default.cf.bosh
port: 5524
roles:
- tag: admin
name: ccadmin
password: c1oudc0w
- tag: admin
name: uaaadmin
password: c1oudc0w
databases:
- tag: cc
name: ccdb
citext: true
- tag: uaa
name: uaadb
citext: true
ccdb: &ccdb
db_scheme: postgres
address: 0.data.default.cf.bosh
port: 5524
roles:
- tag: admin
name: ccadmin
password: c1oudc0w
databases:
- tag: cc
name: ccdb
citext: true
ccdb_ng: *ccdb
uaadb:
db_scheme: postgresql
address: 0.data.default.cf.bosh
port: 5524
roles:
- tag: admin
name: uaaadmin
password: c1oudc0w
databases:
- tag: uaa
name: uaadb
citext: true
cc_api_version: v2
cc: &cc
logging_level: debug
external_host: api
srv_api_uri: http://api.vsc.com
cc_partition: default
db_encryption_key: c1oudc0w
bootstrap_admin_email: admin@vsc.com
bulk_api_password: c1oudc0w
uaa_resource_id: cloud_controller
staging_upload_user: uploaduser
staging_upload_password: c1oudc0w
resource_pool:
resource_directory_key: cc-resources
# Local provider when using NFS
fog_connection:
provider: Local
packages:
app_package_directory_key: cc-packages
droplets:
droplet_directory_key: cc-droplets
default_quota_definition: runaway
ccng: *cc
login:
enabled: false
uaa:
url: http://uaa.vsc.com
spring_profiles: postgresql
no_ssl: true
catalina_opts: -Xmx768m -XX:MaxPermSize=256m
resource_id: account_manager
jwt:
signing_key: |
-----BEGIN RSA PRIVATE KEY-----
MIICXAIBAAKBgQDHFr+KICms+tuT1OXJwhCUmR2dKVy7psa8xzElSyzqx7oJyfJ1
JZyOzToj9T5SfTIq396agbHJWVfYphNahvZ/7uMXqHxf+ZH9BL1gk9Y6kCnbM5R6
0gfwjyW1/dQPjOzn9N394zd2FJoFHwdq9Qs0wBugspULZVNRxq7veq/fzwIDAQAB
AoGBAJ8dRTQFhIllbHx4GLbpTQsWXJ6w4hZvskJKCLM/o8R4n+0W45pQ1xEiYKdA
Z/DRcnjltylRImBD8XuLL8iYOQSZXNMb1h3g5/UGbUXLmCgQLOUUlnYt34QOQm+0
KvUqfMSFBbKMsYBAoQmNdTHBaz3dZa8ON9hh/f5TT8u0OWNRAkEA5opzsIXv+52J
duc1VGyX3SwlxiE2dStW8wZqGiuLH142n6MKnkLU4ctNLiclw6BZePXFZYIK+AkE
xQ+k16je5QJBAN0TIKMPWIbbHVr5rkdUqOyezlFFWYOwnMmw/BKa1d3zp54VP/P8
+5aQ2d4sMoKEOfdWH7UqMe3FszfYFvSu5KMCQFMYeFaaEEP7Jn8rGzfQ5HQd44ek
lQJqmq6CE2BXbY/i34FuvPcKU70HEEygY6Y9d8J3o6zQ0K9SYNu+pcXt4lkCQA3h
jJQQe5uEGJTExqed7jllQ0khFJzLMx0K6tj0NeeIzAaGCQz13oo2sCdeGRHO4aDh
HH6Qlq/6UOV5wP8+GAcCQFgRCcB+hrje8hfEEefHcFpyKH+5g1Eu1k0mLrxK2zd+
4SlotYRHgPCEubokb2S1zfZDWIXW3HmggnGgM949TlY=
-----END RSA PRIVATE KEY-----
verification_key: |
-----BEGIN PUBLIC KEY-----
MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQDHFr+KICms+tuT1OXJwhCUmR2d
KVy7psa8xzElSyzqx7oJyfJ1JZyOzToj9T5SfTIq396agbHJWVfYphNahvZ/7uMX
qHxf+ZH9BL1gk9Y6kCnbM5R60gfwjyW1/dQPjOzn9N394zd2FJoFHwdq9Qs0wBug
spULZVNRxq7veq/fzwIDAQAB
-----END PUBLIC KEY-----
cc:
client_secret: c1oudc0w
admin:
client_secret: c1oudc0w
batch:
username: batchuser
password: c1oudc0w
client:
autoapprove:
- cf
clients:
cf:
override: true
authorized-grant-types: password,implicit,refresh_token
authorities: uaa.none
scope: cloud_controller.read,cloud_controller.write,openid,password.write,cloud_controller.admin,scim.read,scim.write
access-token-validity: 7200
refresh-token-validity: 1209600
scim:
users:
- admin|c1oudc0w|scim.write,scim.read,openid,cloud_controller.admin
- services|c1oudc0w|scim.write,scim.read,openid,cloud_controller.admin
修改cf.yml文件
director_uuid:修改为Bosh Director的UUID,可以通过bosh status查看;
releases的name属性,修改为之前创建CloudFoundry的Release包时输入的包名;
10.58.19.133:修改为可用的Floating IP;
vsc.com:域名,CF平台不能通过IP直接访问使用,必须配置域名,可以自己任意写个域名,然后配置本地DNS Server进行解析即可;
反馈:
root@bosh-cli:~/bosh-workspace/deployments# bosh deployment cf-149.yml
root@bosh-cli:~/bosh-workspace/deployments# bosh deploy
Getting deployment properties from director...
Unable to get properties list from director, trying without it...
Compiling deployment manifest...
Cannot get current deployment information from director, possibly a new deployment
Please review all changes carefully
Deploying `cf-149.yml' to `bosh' (type 'yes' to continue): yes
Director task 10
Preparing deployment
binding deployment (00:00:00)
binding releases (00:00:00)
binding existing deployment (00:00:00)
binding resource pools (00:00:00)
binding stemcells (00:00:00)
binding templates (00:00:01)
binding properties (00:00:00)
binding unallocated VMs (00:00:00)
binding instance networks (00:00:00)
Done 9/9 00:00:01
Preparing package compilation
Preparing DNS
binding DNS (00:00:00)
Done 1/1 00:00:00
Creating bound missing VMs
small/0 (00:00:42)
small/1 (00:00:57)
small/2 (00:01:02)
small/3 (00:01:10)
Done 4/4 00:01:10
Binding instance VMs
data/0 (00:00:01)
dea/0 (00:00:01)
core/0 (00:00:01)
api/0 (00:00:01)
Done 4/4 00:00:01
Preparing configuration
binding configuration (00:00:02)
Done 1/1 00:00:02
Updating job data
data/0 (canary) (00:00:53)
Done 1/1 00:00:53
Updating job core
core/0 (canary) (00:01:08)
Done 1/1 00:01:08
Updating job api
api/0 (canary) (00:01:04)
Done 1/1 00:01:04
Updating job dea
dea/0 (canary) (00:03:20)
Done 1/1 00:03:20
Task 10 done
Started 2013-12-18 11:20:53 UTC
Finished 2013-12-18 11:29:37 UTC
Duration 00:08:44
Deployed `cf-149.yml' to `bosh'
部署过程可能会因为OpenStack的VM虚拟机创建时间过长或PING虚拟机响应过长导致失败,只需要删除Deployments后再重新部署即可。
Deployment删除命令:
root@bosh-cli:~/bosh-workspace/deployments# bosh delete deployment cf
部署成功之后可以使用命令查看虚拟机运行状态:
root@bosh-cli:~/bosh-workspace/deployments# bosh vms
Deployment `cf'
Director task 191
Task 191 done
+-----------+---------+---------------+--------------------------+
| Job/index | State | Resource Pool | IPs |
+-----------+---------+---------------+--------------------------+
| api/0 | running | small | 50.50.0.16, 10.68.19.133 |
| core/0 | running | small | 50.50.0.17 |
| data/0 | running | small | 50.50.0.14 |
| dea/0 | running | small | 50.50.0.15 |
+-----------+---------+---------------+--------------------------+
VMs total: 4
全部Job为Running状态则正常。
6、使用CF客户端管理CloudFoundry运行环境
注意:请先配置DNS Server,对设置的虚拟域名(vsc.com)进行解析,参照文章:《配置DNS Server》和《配置OpenStack环境DNS》
6.1、安装CF命令行工具
gem install cf
6.2、切换CF指向为api.vsc.com
root@bosh-cli:~# cf target api.vsc.com
Setting target to http://api.vsc.com... OK
target: http://api.vsc.com
organization: system_domain
6.3、登陆到CF平台,本环境账号密码为admin/c1oudc0w
root@bosh-cli:~# cf login admin
target: http://api.vsc.com
Password> ********
Authenticating... OK
There are no spaces. You may want to create one with create-space.
6.4、创建组织
root@bosh-cli:~# cf create-org venusource
Creating organization venusource... OK
Switching to organization venusource... OK
There are no spaces. You may want to create one with create-space.
6.5、创建并切换空间
root@bosh-cli:~# cf create-space development
Creating space development... OK
Adding you as a manager... OK
Adding you as a developer... OK
Space created! Use `cf switch-space development` to target it.
root@bosh-cli:~# cf switch-space development
Switching to space development... OK
target: http://api.vsc.com
organization: venusource
space: development
6.6、下载测试Ruby工程
root@bosh-cli:~# git clone https://github.com/cloudfoundry-community/cf_demoapp_ruby_rack.git
6.7、打包并发布测试工程
root@bosh-cli:~# cd cf_demoapp_ruby_rack/
root@bosh-cli:~/cf_demoapp_ruby_rack# rm manifest.yml
root@bosh-cli:~/cf_demoapp_ruby_rack# bundle
Resolving dependencies...
Using rack (1.5.2)
Using bundler (1.3.5)
Your bundle is complete!
Use `bundle show [gemname]` to see where a bundled gem is installed.
root@bosh-cli:~/cf_demoapp_ruby_rack# cf push
Name> hello
Instances> 1
1: 128M
2: 256M
3: 512M
4: 1G
Memory Limit> 1
Creating hello... OK
1: hello
2: none
Subdomain> hello
1: vsc.com
2: none
Domain> mycloud.com
Creating route hello.mycloud.com... OK
Binding hello.mycloud.com to hello... OK
Create services for application?> n
Save configuration?> n
Uploading hello... OK
Starting hello... OK
-----> Downloaded app package (4.0K)
....
Checking hello...
Staging in progress...
0/1 instances: 1 starting
0/1 instances: 1 starting
0/1 instances: 1 starting
1/1 instances: 1 running
OK
部署完毕!欢迎各位交流指正,谢谢!