OpenStack对象存储服务swift多节点安装(VirtualBoxmulti-host)
LIUYulong
刘宇龙
目录
0.基本结构...3
1. 虚拟机的创建... 3
1.1 代理节点虚拟机proxy-node. 3
1.2 五个存储节点 swift-Storage-server.4
2.网络配置...5
2.1 proxy节点... 5
2.2各个存储节点...5
3.各个节点安装的公共软件和操作...6
3.1公共部分的软件安装...6
3.2各个节点的公共操作...7
4.proxy节点的安装和操作...8
4.1安装proxy服务...8
4.2生成ssh-key.8
4.3配置memcached服务...8
4.4 配置proxy服务/etc/swift/proxy-server.conf,使用keystone认证...8
4.5创建account, container 以及object环(ring)... 10
4.6添加各个存储节点到环中...10
4.7验证环...10
4.8平衡环...11
4.9复制4.8生成的account.ring.gz, container.ring.gz, and object.ring.gz到 Proxy 和各个Storage 节点中的目录/etc/swift.11
4.10确保swift用户对/etc/swift目录的权限...11
4.11启动proxy服务...12
5.storage存储节点的安装和操作...12
5.1相关软件的安装...12
5.2给swift分配新硬盘...12
5.3 创建并设置/etc/rsyncd.conf12
5.4设置/etc/default/rsync.13
5.5启动rsync.13
5.6设置account-server服务 /etc/swift/account-server.conf13
5.7设置container-server服务/etc/swift/container-server.conf14
5.8设置object-server 服务/etc/swift/object-server.conf14
6.安装keystone参考其他文档...15
7.验证安装...22
7.1查看租户状态...22
7.2上传文件到swift.22
7.3查看各个存储节点的对象分布...23
0.基本结构
1. 虚拟机的创建
1.1 代理节点虚拟机proxy-node
(1)双网卡
(2)高计算性能 2核心cpu
(3)一个硬盘 40G
1.2 五个存储节点 swift-Storage-server
(1)单网卡
(2)计算性能要求不高 1核心cpu
(3)两个40G硬盘 系统盘和swift数据盘
1.2各个服务器系统安装
选择ubuntu-12.04.2-desktop-amd64.iso
2.网络配置
2.1 proxy节点
删除ununtu的网络管理器network-manager
sudo apt-get purge network-manager
配置network网络管理器
sudo vim /etc/network/interfaces
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
auto eth1
iface eth1 inet static
address 172.16.0.1
netmask 255.255.0.0
network 172.16.0.0
broadcast 172.16.255.255
重启网络
sudo /etc/init.d/networking restart
2.2各个存储节点
如图:四个存储节点配置在和代理节点172.16.xxx.xxx。
这里设置的IP分别为:
swift-Storage-server-01:172.16.0.2
swift-Storage-server-02:172.16.0.3
swift-Storage-server-03:172.16.0.4
swift-Storage-server-04:172.16.0.5
删除ununtu的网络管理器network-manager
sudo apt-get purge network-manager
配置network网络管理器
sudo vim /etc/network/interfaces
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet dhcp
auto eth1
iface eth1 inet static
address 172.16.0.2
netmask 255.255.0.0
network 172.16.0.0
boradcast 172.16.255.255
重启网络
sudo /etc/init.d/networking restart
3.各个节点安装的公共软件和操作
3.1公共部分的软件安装
(1)最新版的SAIO - Swift AllIn One
apt-get update
apt-get install curl gcc memcached rsync sqlite3 xfsprogs git-core libffi-dev python-setuptools
apt-get install python-coverage python-dev python-nose python-simplejson python-xattr python-eventlet python-greenlet python-pastedeploy python-netifaces python-pip python-dnspython python-mock python-mysqldbsudo apt-get install -y python-software-properties
(2)老版本的SAIO - Swift AllIn One
sudo apt-get install -y python-software-properties
sudo add-apt-repository -y ppa:swift-core/release
sudo apt-get update
sudo apt-get install -y curl gcc git-core memcached python-coverage python-dev python-nose python-setuptools python-simplejson python-xattr sqlite3 xfsprogs python-eventlet python-greenlet python-pastedeploy python-netifaces python-pip
sudo pip install mock
sudo apt-get install -y screen ssh vim
(3)官方网站的多节点安装
apt-get install python-software-properties
add-apt-repository ppa:swift-core/release
apt-get update
apt-get install swift python-swiftclient openssh-server
3.2各个节点的公共操作
(1)给swift用户目录权限
mkdir -p /etc/swift
chown -R swift:swift /etc/swift/
(2)随机生成swift.conf
cat >/etc/swift/swift.conf <<EOF
[swift-hash]
# random unique strings that can never change (DO NOT LOSE)
swift_hash_path_prefix = `od -t x8 -N 8 -A n </dev/random`
swift_hash_path_suffix = `od -t x8 -N 8 -A n </dev/random`
EOF
(3) 拷贝(2)生成的swift.conf到所有节点的/etc/swift/目录,包括proxy-node和storage-node
scp firstnode.example.com:/etc/swift/swift.conf /etc/swift/
(4)给swift用户/var/run/swift目录权限
mkdir -p /var/run/swift
chown swift:swift /var/run/swift
4.proxy节点的安装和操作
4.1安装proxy服务
apt-get install swift-proxy memcached
4.2生成ssh-key
cd /etc/swift
openssl req -new -x509 -nodes -out cert.crt -keyout cert.key
4.3配置memcached服务
perl -pi -e "s/-l 127.0.0.1/-l $PROXY_LOCAL_NET_IP/" /etc/memcached.conf
service memcached restart
4.4 配置proxy服务/etc/swift/proxy-server.conf,使用keystone认证
[DEFAULT]
#cert_file = /etc/swift/cert.crt
#key_file = /etc/swift/cert.key
bind_port = 8888
bind_ip=172.16.0.1
user = swift
log_facility = LOG_LOCAL1
#[pipeline:main]
#pipeline = healthcheck cache tempauth proxy-logging proxy-server
[pipeline:main]
pipeline = healthcheck cache authtoken keystone proxy-server
[app:proxy-server]
use = egg:swift#proxy
allow_account_management = true
account_autocreate = true
#[filter:tempauth]
#use = egg:swift#tempauth
#user_admin_admin = admin .admin .reseller_admin
#user_test_tester = testing .admin
#user_test2_tester2 = testing2 .admin
#user_test_tester3 = testing3
[filter:keystone]
paste.filter_factory = keystone.middleware.swift_auth:filter_factory
operator_roles = Member,admin, swiftoperator
[filter:authtoken]
paste.filter_factory = keystone.middleware.auth_token:filter_factory
# Delaying the auth decision is required to support token-less
# usage for anonymous referrers ('.r:*').
delay_auth_decision = 10
service_port = 5000
service_host = 172.16.0.1
auth_port = 35357
auth_host = 172.16.0.1
auth_protocol = http
auth_uri = http://172.16.0.1:5000/
auth_token = 0123456789TOKEN987654231
admin_token = 0123456789TOKEN987654231
admin_tenant_name = service
admin_user = swift
admin_password = swiftpass
[filter:healthcheck]
use = egg:swift#healthcheck
[filter:cache]
use = egg:swift#memcache
[filter:proxy-logging]
use = egg:swift#proxy_logging
4.5创建account, container 以及object环(ring)
cd /etc/swift
swift-ring-builder account.builder create 18 3 1
swift-ring-builder container.builder create 18 3 1
swift-ring-builder object.builder create 18 3 1
4.6添加各个存储节点到环中
4个节点
172.16.0.2-172.16.0.5
swift-ring-builder account.builder add z1-172.16.0.2:6002/sdb1 100
swift-ring-builder container.builder add z1-172.16.0.2:6001/sdb1 100
swift-ring-builder object.builder add z1-172.16.0.2:6000/sdb1 100
swift-ring-builder account.builder add z2-172.16.0.3:6002/sdb1 100
swift-ring-builder container.builder add z2-172.16.0.3:6001/sdb1 100
swift-ring-builder object.builder add z2-172.16.0.3:6000/sdb1 100
swift-ring-builder account.builder add z3-172.16.0.4:6002/sdb1 100
swift-ring-builder container.builder add z3-172.16.0.4:6001/sdb1 100
swift-ring-builder object.builder add z3-172.16.0.4:6000/sdb1 100
swift-ring-builder account.builder add z4-172.16.0.5:6002/sdb1 100
swift-ring-builder container.builder add z4-172.16.0.5:6001/sdb1 100
swift-ring-builder object.builder add z4-172.16.0.5:6000/sdb1 100
4.7验证环
swift-ring-builder account.builder
swift-ring-builder container.builder
swift-ring-builder object.builder
结果如下:
4.8平衡环
swift-ring-builder account.builder rebalance
swift-ring-builder container.builder rebalance
swift-ring-builder object.builder rebalance
4.9复制4.8生成的account.ring.gz,container.ring.gz, and object.ring.gz到 Proxy 和各个Storage 节点中的目录/etc/swift.
scp 172.16.0.1:/etc/swift/*.gz /etc/swift/
4.10确保swift用户对/etc/swift目录的权限
chown -R swift:swift /etc/swift
4.11启动proxy服务
swift-init proxy start
5.storage存储节点的安装和操作
5.1相关软件的安装
apt-get install swift-account swift-container swift-object xfsprogs
5.2给swift分配新硬盘
fdisk /dev/sdb
n 写入分区
p 主分区
e扩展分区
w 写入分区表退出
mkfs.xfs -i size=512 /dev/sdb1
echo "/dev/sdb1 /srv/node/sdb1 xfs noatime,nodiratime,nobarrier,logbufs=8 0 0" >> /etc/fstab
mkdir -p /srv/node/sdb1
mount /srv/node/sdb1
chown swift:swift /srv/node/sdb1
5.3创建并设置/etc/rsyncd.conf
cat >/etc/rsyncd.conf <<EOF
uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 172.16.0.2-172.16.0.5
[account]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/account.lock
[container]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/container.lock
[object]
max connections = 2
path = /srv/node/
read only = false
lock file = /var/lock/object.lock
EOF
5.4设置/etc/default/rsync
perl -pi -e 's/RSYNC_ENABLE=false/RSYNC_ENABLE=true/' /etc/default/rsync
5.5启动rsync
service rsync start
5.6设置account-server服务 /etc/swift/account-server.conf
cat >/etc/swift/account-server.conf <<EOF
[DEFAULT]
bind_ip = 172.16.0.2-172.16.0.5
workers = 2
[pipeline:main]
pipeline = account-server
[app:account-server]
use = egg:swift#account
[account-replicator]
[account-auditor]
[account-reaper]
EOF
5.7设置container-server服务/etc/swift/container-server.conf
cat >/etc/swift/container-server.conf <<EOF
[DEFAULT]
bind_ip = 172.16.0.2-172.16.0.5
workers = 2
[pipeline:main]
pipeline = container-server
[app:container-server]
use = egg:swift#container
[container-replicator]
[container-updater]
[container-auditor]
[container-sync]
EOF
5.8设置object-server服务/etc/swift/object-server.conf
cat >/etc/swift/object-server.conf <<EOF
[DEFAULT]
bind_ip = 172.16.0.2-172.16.0.5
workers = 2
[pipeline:main]
pipeline = object-server
[app:object-server]
use = egg:swift#object
[object-replicator]
[object-updater]
[object-auditor]
EOF
5.9启动各个存储节点的各个服务
swift-init all start
6.安装keystone参考其他文档
7.验证安装
7.1查看租户状态
swift -v -V 2.0 -A http://172.16.0.1:5000/v2.0 -U openstackDemo:adminUser -K secretword stat
结果如下:
7.2上传文件到swift
swift -v -V 2.0 -A http://172.16.0.1:5000/v2.0 -U openstackDemo:adminUser -K secretword nihao ./insall.txt
7.3查看各个存储节点的对象分布