openstack Ocata安装（Object Storage service overview）

OpenStack对象存储（swift）是一个多租户的对象存储系统，它支持大规模扩展，可以以低成本来管理大型的非结构化数据。
二、安装并配置控制器节点
获得 admin 凭证来获取只有管理员能执行的命令的访问权限

# source /home/admin-openrc.sh

要创建身份认证服务的凭证有这几个步骤：创建 swift 用户，给 swift 用户添加 admin 角色，创建 swift 服务条目，创建对象存储服务 API 端点。

# openstack user create --domain default --password-prompt swift
# openstack role add --project service --user swift admin
# openstack service create --name swift --description "OpenStack Object Storage" object-store
# openstack endpoint create --region RegionOne object-store public http://172.16.5.135:8080/v1/AUTH_%\(tenant_id\)s
# openstack endpoint create --region RegionOne object-store internal http://172.16.5.135:8080/v1/AUTH_%\(tenant_id\)s
# openstack endpoint create --region RegionOne object-store admin http://172.16.5.135:8080/v1

三、配置组件

安装软件包

# yum install openstack-swift-proxy python-swiftclient python-keystoneclient python-keystonemiddleware memcached

从对象存储的仓库源中获取代理服务的配置文件

# curl -o /etc/swift/proxy-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/proxy-server.conf-sample?h=stable/ocata

编辑文件 /etc/swift/proxy-server.conf

# vi /etc/swift/proxy-server.conf

[DEFAULT]
bind_port = 8080
user = swift
swift_dir = /etc/swift
[pipeline:main]
pipeline = catch_errors gatekeeper healthcheck proxy-logging cache container_sync bulk ratelimit authtoken keystoneauth copy container-quotas account-quotas slo dlo versioned_writes proxy-logging proxy-server
[app:proxy-server]
use = egg:swift#proxy
account_autocreate = True
[filter:keystoneauth]
use = egg:swift#keystoneauth
operator_roles = admin,user
[filter:authtoken]
paste.filter_factory = keystonemiddleware.auth_token:filter_factory
auth_uri = http://172.16.5.135:5000
auth_url = http://172.16.5.13535357
memcached_servers = 172.16.5.135:11211
auth_type = password
project_domain_name = default
user_domain_name = default
project_name = service
username = swift
password = 123qwe
delay_auth_decision = True
[filter:cache]
use = egg:swift#memcache
memcache_servers = controller:11211

四、安装和配置存储节点（每个存储节点都要执行）
安装支持的工具包：

# yum install xfsprogs rsync

使用XFS格式化/dev/sdb和/dev/sdc设备：

# mkfs.xfs /dev/sdb
# mkfs.xfs /dev/sdc

创建挂载点目录结构：

# mkdir -p /srv/node/sdb
# mkdir -p /srv/node/sdc

编辑/etc/fstab文件并添加以下内容：

/dev/sdb /srv/node/sdb xfs noatime,nodiratime,nobarrier,logbufs=8 0 2
/dev/sdc /srv/node/sdc xfs noatime,nodiratime,nobarrier,logbufs=8 0 2

挂载设备：

# mount /srv/node/sdb
# mount /srv/node/sdc

创建并编辑/etc/rsyncd.conf文件并包含以下内容：

uid = swift
gid = swift
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
address = 172.16.5.135

[account]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/account.lock

[container]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/container.lock

[object]
max connections = 2
path = /srv/node/
read only = False
lock file = /var/lock/object.lock

启动 “rsyncd” 服务和配置它随系统启动：

# systemctl enable rsyncd.service
# systemctl start rsyncd.service

五、配置组件（每个存储节点都要执行，bind_ip是本机IP）
安装软件包：

# yum install openstack-swift-account openstack-swift-container openstack-swift-object

从对象存储源仓库中获取accounting, container以及object服务配置文件

# curl -o /etc/swift/account-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/account-server.conf-sample?h=stable/ocata
# curl -o /etc/swift/container-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/container-server.conf-sample?h=stable/ocata
# curl -o /etc/swift/object-server.conf https://git.openstack.org/cgit/openstack/swift/plain/etc/object-server.conf-sample?h=stable/ocata

编辑 /etc/swift/account-server.conf 文件:

#vi /etc/swift/account-server.conf

[DEFAULT]
bind_ip = 172.16.5.135
bind_port = 6202
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True

[pipeline:main]
pipeline = healthcheck recon account-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

编辑/etc/swift/container-server.conf文件：

# vi /etc/swift/container-server.conf

[DEFAULT]
bind_ip = 172.16.5.135
bind_port = 6201
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True

[pipeline:main]
pipeline = healthcheck recon container-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift

编辑/etc/swift/object-server.conf文件：

# vi /etc/swift/object-server.conf

[DEFAULT]
bind_ip = 172.16.5.135
bind_port = 6200
user = swift
swift_dir = /etc/swift
devices = /srv/node
mount_check = True

[pipeline:main]
pipeline = healthcheck recon object-server

[filter:recon]
use = egg:swift#recon
recon_cache_path = /var/cache/swift
recon_lock_path = /var/lock

确认挂载点目录结构是否有合适的所有权：

# chown -R swift:swift /srv/node

创建 “recon” 目录和确保它有合适的所有权：

# mkdir -p /var/cache/swift
# chown -R root:swift /var/cache/swift
# chmod -R 775 /var/cache/swift

六、创建，分发并初始化rings（每个存储节点都要执行）
创建账户ring，切换到 /etc/swift目录创建基本 account.builder 文件。

# swift-ring-builder account.builder create 10 3 1

添加每个节点到 ring 中:

# swift-ring-builder account.builder add --region 1 --zone 1 --ip 172.16.5.135 --port 6202 --device sdb --weight 100
# swift-ring-builder account.builder add --region 1 --zone 1 --ip 172.16.5.135 --port 6202 --device sdc --weight 100

# swift-ring-builder account.builder add --region 1 --zone 2 --ip 172.16.5.136 --port 6202 --device sdb --weight 100
# swift-ring-builder account.builder add --region 1 --zone 2 --ip 172.16.5.136 --port 6202 --device sdc --weight 100

平衡 ring：

 # swift-ring-builder account.builder rebalance

验证 ring 的内容：

# swift-ring-builder account.builder

account.builder, build version 4
1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 1
The overload factor is 0.00% (0.000000)
Devices:    id  region  zone      ip address  port  replication ip  replication port      name weight partitions balance meta
             0       1     1       172.16.5.135  6202       10.0.0.51              6202      sdb  100.00          0 -100.00
             1       1     1       172.16.5.135  6202       10.0.0.51              6202      sdc  100.00          0 -100.00
             2       1     2       172.16.5.136  6202       10.0.0.52              6202      sdb  100.00          0 -100.00
             3       1     2       172.16.5.136  6202       10.0.0.52              6202      sdc  100.00          0 -100.00

创建容器ring，切换到 /etc/swift目录创建基本container.builder文件：

# swift-ring-builder container.builder create 10 3 1

添加每个节点到 ring 中：

# swift-ring-builder container.builder add \
  --region 1 --zone 1 --ip 172.16.5.135 --port 6201 --device sdb --weight 100
# swift-ring-builder container.builder add \
  --region 1 --zone 1 --ip 172.16.5.135 --port 6201 --device sdc --weight 100

# swift-ring-builder container.builder add \
  --region 1 --zone 2 --ip 172.16.5.136 --port 6201 --device sdb --weight 100
# swift-ring-builder container.builder add \
  --region 1 --zone 2 --ip 172.16.5.136 --port 6201 --device sdc --weight 100

平衡 ring：

# swift-ring-builder container.builder rebalance

验证 ring 的内容：

# swift-ring-builder container.builder

container.builder, build version 4
1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 1
The overload factor is 0.00% (0.000000)
Devices:    id  region  zone      ip address  port  replication ip  replication port      name weight partitions balance meta
             0       1     1       172.16.5.135  6201       10.0.0.51              6201      sdb  100.00          0 -100.00
             1       1     1       172.16.5.135  6201       10.0.0.51              6201      sdc  100.00          0 -100.00
             2       1     2       172.16.5.136  6201       10.0.0.52              6201      sdb  100.00          0 -100.00
             3       1     2       172.16.5.136  6201       10.0.0.52              6201      sdc  100.00          0 -100.00

创建对象ring，切换到 /etc/swift目录创建基本object.builder文件：

# swift-ring-builder object.builder create 10 3 1

添加每个节点到 ring 中：

# swift-ring-builder object.builder add \
  --region 1 --zone 1 --ip 172.16.5.135 --port 6200 --device sdb --weight 100
# swift-ring-builder object.builder add \
  --region 1 --zone 1 --ip 172.16.5.135 --port 6200 --device sdc --weight 100

# swift-ring-builder object.builder add \
  --region 1 --zone 2 --ip 172.16.5.136 --port 6200 --device sdb --weight 100
# swift-ring-builder object.builder add \
  --region 1 --zone 2 --ip 172.16.5.136 --port 6200 --device sdc --weight 100

平衡 ring：

# swift-ring-builder object.builder rebalance

验证 ring 的内容：

# swift-ring-builder object.builder

object.builder, build version 4
1024 partitions, 3.000000 replicas, 1 regions, 2 zones, 4 devices, 100.00 balance, 0.00 dispersion
The minimum number of hours before a partition can be reassigned is 1
The overload factor is 0.00% (0.000000)
Devices:    id  region  zone      ip address  port  replication ip  replication port      name weight partitions balance meta
             0       1     1       172.16.5.135  6200       10.0.0.51              6200      sdb  100.00          0 -100.00
             1       1     1       172.16.5.135  6200       10.0.0.51              6200      sdc  100.00          0 -100.00
             2       1     2       172.16.5.136  6200       10.0.0.52              6200      sdb  100.00          0 -100.00
             3       1     2       172.16.5.136  6200       10.0.0.52              6200      sdc  100.00          0 -100.00

七、完成安装
从对象存储源仓库中获取 /etc/swift/swift.conf 文件：

# curl -o /etc/swift/swift.conf \
  https://git.openstack.org/cgit/openstack/swift/plain/etc/swift.conf-sample?h=stable/ocata

编辑 /etc/swift/swift.conf 文件

[swift-hash]
swift_hash_path_suffix = 123
swift_hash_path_prefix = qwe

[storage-policy:0]
name = Policy-0
default = yes

复制swift.conf 文件到每个存储节点和其他允许了代理服务的额外节点的 /etc/swift 目录。
在所有节点上，确认配置文件目录是否有合适的所有权：

# chown -R root:swift /etc/swift

在控制节点和其他运行了代理服务的节点上，启动对象存储代理服务及其依赖服务，并将它们配置为随系统启动：

# systemctl enable openstack-swift-proxy.service memcached.service
# systemctl start openstack-swift-proxy.service memcached.service

在存储节点上，启动对象存储服务，并将其设置为随系统启动：

# systemctl enable openstack-swift-account.service openstack-swift-account-auditor.service \
  openstack-swift-account-reaper.service openstack-swift-account-replicator.service
# systemctl start openstack-swift-account.service openstack-swift-account-auditor.service \
  openstack-swift-account-reaper.service openstack-swift-account-replicator.service
# systemctl enable openstack-swift-container.service \
  openstack-swift-container-auditor.service openstack-swift-container-replicator.service \
  openstack-swift-container-updater.service
# systemctl start openstack-swift-container.service \
  openstack-swift-container-auditor.service openstack-swift-container-replicator.service \
  openstack-swift-container-updater.service
# systemctl enable openstack-swift-object.service openstack-swift-object-auditor.service \
  openstack-swift-object-replicator.service openstack-swift-object-updater.service
# systemctl start openstack-swift-object.service openstack-swift-object-auditor.service \
  openstack-swift-object-replicator.service openstack-swift-object-updater.service

八、 验证操作
如果其中的一项或多项步骤没有正确执行，请在/var/log/audit/audit.log文件中检查SELinux的关于禁止swift过程的信息。如果该文件存在的话，将/srv/node目录下swift_data_t type, object_r role 和the system_u user关于安全等级的信息设置成最低安全等级（s0）

# chcon -R system_u:object_r:swift_data_t:s0 /srv/node

导入demo凭证：

# source /home/demo-openrc.sh

显示服务状态：

# swift --debug stat

进入仪表盘（控制界面），去上传文件试一下吧。

注：
打开swift日志

# vi /etc/rsyslog.d/openstack-swift.conf

local0.*;local2.*       /var/log/swift/swift.log
#&                        stop

# systemctl restart rsyslog.service

openstack swift做glance后端存储

# vim /etc/glance/glance-api.conf

[glance_store]

default_store = swift          默认为file 改为swift

stores = file, http                默认不用动

swift_store_auth_version = 2        默认版本为2

stores=glance.store.swift.Store,glance.store.filesystem.Store      此项一定要加上，不然无法上传

swift_store_auth_address = http://controller:5000/v2.0    controller的keystone认证

swift_store_user = service:swift    使用swift用户

swift_store_key = swift        密码

swift_store_container = glance        将要被创建的容器

swift_store_create_container_on_put = True        上传开

swift_store_large_object_size = 5120        最大5G限制，但与glance结合后限制无效

swift_store_large_object_chunk_size = 200        最大200个容器

swift_enable_snet = False

# systemctl restart openstack-glance-api

九、排查错误
如果你很不幸遇到了503错误，并且根据日志的报错内容觉得是代理服务可能出错了，但是无论如何修改依旧报503错误。亲，你创建挂载点目录结构要修改成下面的：

# mkdir -p /srv/node/sdb
# mkdir -p /srv/node/sdc

并且ring的时候：–device sdb –device sdc。