ceph 对象存储

最新推荐文章于 2024-04-11 20:44:23 发布

weixin_34185364

最新推荐文章于 2024-04-11 20:44:23 发布

阅读量286

点赞数

文章标签： swift 运维 python

原文链接：https://my.oschina.net/wangzilong/blog/1596690

版权

2019独角兽企业重金招聘Python工程师标准>>>

[TOC]

ceph对象存储

1 准备

准备好虚机的相关配置和网络 ##1.1 准备一台虚机准备一台虚机，您关闭防火墙，关闭是seinux，且能拼通ceph集群管理节点

[root@host-172-18-1-125 ~]# systemctl status iptables.service
iptables.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)

[root@host-172-18-1-125 ~]# getenforce
Disabled
[root@host-172-18-1-125 ~]# service iptables status
Redirecting to /bin/systemctl status iptables.service
iptables.service
   Loaded: not-found (Reason: No such file or directory)
   Active: inactive (dead)

[root@host-172-18-1-125 ~]#

[root@ops-controller tmp]# ping 172.18.1.125
PING 172.18.1.125 (172.18.1.125) 56(84) bytes of data.
64 bytes from 172.18.1.125: icmp_seq=1 ttl=63 time=0.521 ms
64 bytes from 172.18.1.125: icmp_seq=2 ttl=63 time=0.398 ms
64 bytes from 172.18.1.125: icmp_seq=3 ttl=63 time=0.571 ms
^C
--- 172.18.1.125 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.398/0.496/0.571/0.077 ms

1.2 设置到ceph管理节点的免密登录

1.2.1 配置好/etc/hosts

配置准备安装对象网关的虚机主机名

[root@host-172-18-1-125 ~]# hostname
radosgw02

ceph 管理节点在/etc/hosts文件里面写入IP的解析

[root@ops-controller tmp]# cat /etc/hosts|grep 125
172.18.1.125 radosgw02

配置ceph管理节点到网关节点的免密登录

[root@ops-controller tmp]# ssh-copy-id -i ~/.ssh/id_rsa.pub radosgw02
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
The authenticity of host 'radosgw02 (172.18.1.125)' can't be established.
ECDSA key fingerprint is SHA256:JcA42ovBK8I4uMYipRDJ3H49c0clrWM5wxxNsdqBMIo.
ECDSA key fingerprint is MD5:e1:67:0c:f5:b1:d5:7a:82:48:1b:40:05:b9:0a:36:95.
Are you sure you want to continue connecting (yes/no)? yes
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@radosgw02's password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh 'radosgw02'"
and check to make sure that only the key(s) you wanted were added.

检查是否可以免密登录

[root@ops-controller tmp]# ssh radosgw02
Last login: Fri Dec 29 00:45:15 2017 from 172.18.0.80
[root@radosgw02 ~]#
#可以免密登录

2 安装ceph对象网关

通过ceph管理节点上的ceph工具ceph-deploy 远程给网关节点安装对象网关。 ##2.1 远程安装对象网关

[root@ops-controller ceph]# ceph-deploy install --rgw radosgw02

2.2 设置网关节点的管理权限

[root@ops-controller ceph]# ceph-deploy admin radosgw02

2.3 新卷网关实例

[root@ops-controller ceph]# ceph-deploy rgw create radosgw02

3 测试网关

3.1 测试s3接口

3.1.1 在网关上为s3创建用户

[root@radosgw ~]# radosgw-admin user create --uid="wangzilong" --display-name="User For Wangzilong"
2017-12-29 01:27:30.099661 7f0c942db9c0 0 WARNING: detected a version of libcurl which contains a bug in curl_multi_wait(). enabling a workaround that may degrade performance slightly.
{
    "user_id": "wangzilong",
    "display_name": "User For Wangzilong",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [],
    "keys": [
        {
            "user": "wangzilong",
            "access_key": "Y8SGW0272XILVFMOXQK3",
            "secret_key": "P7rjPy47iAP7ZjkpYERy70EMKDDVlHVGHSAyLnO0"
        }
    ],
    "swift_keys": [],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    },
    "temp_url_keys": []
}

3.1.2 通过s3cmd客户端验证s3接口的可用性

3.1.2.1 下载安装s3cmd 客户端

^C[root@radosgw ceph]# yum list|grep s3cmd
s3cmd.noarch 1.6.1-1.el7 @epel
[root@radosgw ceph]# yum install s3cmd.noarch -y

3.1.2.2 配置s3cmd

[root@radosgw ceph]# s3cmd --configure

Enter new values or accept defaults in brackets with Enter.
Refer to user manual for detailed description of all options.

Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
Access Key [62V7BQ3XK6VY0YGHC27X]: #这里输入上面创建用户返回文件中的 Y8SGW0272XILVFMOXQK3
Secret Key [t2qKSwoO7HJrrdbl3clBxbJ03thHOKYs0t4x8RMS]: #这里输入上面的 P7rjPy47iAP7ZjkpYERy70EMKDDVlHVGHSAyLnO0
Default Region [US]:

Encryption password is used to protect your files from reading
by unauthorized persons while in transfer to S3
Encryption password:
Path to GPG program [/usr/bin/gpg]:

When using secure HTTPS protocol all communication with Amazon S3
servers is protected from 3rd party eavesdropping. This method is
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
Use HTTPS protocol [No]: #这里输入no

On some networks all internet access must go through a HTTP proxy.
Try setting it here if you can't connect to S3 directly
HTTP Proxy server name:

New settings:
Access Key: 62V7BQ3XK6VY0YGHC27X
Secret Key: t2qKSwoO7HJrrdbl3clBxbJ03thHOKYs0t4x8RMS
Default Region: US
Encryption password:
Path to GPG program: /usr/bin/gpg
Use HTTPS protocol: False
HTTP Proxy server name:
HTTP Proxy server port: 0

Test access with supplied credentials? [Y/n] n #这里输入n

Save settings? [y/N] y # 输入y保存
Configuration saved to '/root/.s3cfg'

3.1.3 验证s3接口

3.1.3.1 创建bucket

[root@radosgw ~]# s3cmd mb s3://bucket_20171229
Bucket 's3://bucket_20171229/' created

3.1.3.2 查询bucket列表

[root@radosgw ~]# s3cmd ls
2017-12-28 09:39 s3://Ldl
2017-12-29 06:21 s3://bucket_20171229
2017-12-28 09:03 s3://ldl

3.1.3.3 bucket注入文件

[root@radosgw ~]# s3cmd put /etc/hosts s3://bucket_20171229
upload: '/etc/hosts' -> 's3://bucket_20171229/hosts' [1 of 1]
158 of 158 100% in 0s 3.32 kB/s done
[root@radosgw ~]# s3cmd ls s3://bucket_20171229
2017-12-29 06:22 158 s3://bucket_20171229/hosts

3.1.3.4 删除文件

[root@radosgw ~]# s3cmd del s3://bucket_20171229/hosts
delete: 's3://bucket_20171229/hosts'
[root@radosgw ~]# s3cmd ls s3://bucket_20171229

3.1.3.5 删除bucket

[root@radosgw ~]# s3cmd rb s3://bucket_20171229
Bucket 's3://bucket_20171229/' removed
[root@radosgw ~]# s3cmd ls
2017-12-28 09:39 s3://Ldl
2017-12-28 09:03 s3://ldl

3.2 测试swift接口

3.2.1 在网关上为swift创建子用户

[root@radosgw ceph]# radosgw-admin subuser create --uid=wangzilong --subuser=wangzilong:swift --access=full
2017-12-29 01:36:15.279806 7f00149fe9c0 0 WARNING: detected a version of libcurl which contains a bug in curl_multi_wait(). enabling a workaround that may degrade performance slightly.
{
    "user_id": "wangzilong",
    "display_name": "User For Wangzilong",
    "email": "",
    "suspended": 0,
    "max_buckets": 1000,
    "auid": 0,
    "subusers": [
        {
            "id": "wangzilong:swift",
            "permissions": "full-control"
        }
    ],
    "keys": [
        {
            "user": "wangzilong",
            "access_key": "Y8SGW0272XILVFMOXQK3",
            "secret_key": "P7rjPy47iAP7ZjkpYERy70EMKDDVlHVGHSAyLnO0"
        }
    ],
    "swift_keys": [
        {
            "user": "wangzilong:swift",
            "secret_key": "pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC"
        }
    ],
    "caps": [],
    "op_mask": "read, write, delete",
    "default_placement": "",
    "placement_tags": [],
    "bucket_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    },
    "user_quota": {
        "enabled": false,
        "max_size_kb": -1,
        "max_objects": -1
    },
    "temp_url_keys": []
}

3.2.2 安装swift相关的Python插件

sudo yum install python-setuptools
sudo easy_install pip
sudo pip install --upgrade setuptools
sudo pip install --upgrade python-swiftclient

3.2.3 验证swift

swift -A http://{IP ADDRESS}:{port}/auth/1.0 -U testuser:swift -K '{swift_secret_key}' list

3.2.3.1 查看bucket列表

list

[root@radosgw ceph]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' list
[root@radosgw ceph]#

3.2.3.2 创建bucket

post

[root@radosgw ceph]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' post bucket_swift
[root@radosgw ceph]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' list
bucket_swift

3.2.3.3 删除bucket

delete

[root@radosgw ceph]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' list
bucket_swift
second_swift
[root@radosgw ceph]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' delete second_swift
second_swift
[root@radosgw ceph]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' list
bucket_swift
[root@radosgw ceph]#

3.2.3.4 bucket 中上传文件

upload

[root@radosgw ceph]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' upload bucket_swift /etc/hosts
etc/hosts
[root@radosgw ceph]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' list bucket_swift
etc/hosts

3.2.3.5 从bucket下载文件

download

[root@radosgw opt]# swift -A http://172.18.1.113:7480/auth/1.0 -U wangzilong:swift -K 'pJCr2Wny71GEPCgpDdU3OaOVENbXhBGkt5P9BTRC' download -D /opt bucket_swift
/opt/etc/hosts [auth 0.004s, headers 0.011s, total 0.011s, 0.022 MB/s]
[root@radosgw opt]# ll /opt/etc/hosts
-rw-r--r-- 1 root root 158 Jun 7 2013 /opt/etc/hosts

转载于:https://my.oschina.net/wangzilong/blog/1596690