如何部署使用Intel开源的对象存储测试套件--CosBench-CSDN博客

##1.引言

我最近一直使用Ceph的对象存储，测试方法也是五花八门，有自己用Python写过测试脚本，也有用C++写过。觉得既然S3是一个标准的、通用的对象存储服务，应该有一个测试套件来进行测试。后来在社区中看到大家采用Intel的开源软件Cosbench进行测试，于是我也进行了部署及使用。

##2.部署过程

###2.1 机器准备我使用了七台机器，host1~host7 角色： host1：兼任controller driver host2~6 drvier

###2.2 安装包部署将Cosbench安装包分别解压到各自的记录，安装包可以在如下地址下载： Cosbench说明

2.3 基本信息配置

Controller配置在 install-folder/conf中找到controller.conf配置文件，将内容修改如下：

[controller]
drivers = 7
log_level = INFO
log_file = log/system.log
archive_dir = archive

[driver1]
name = driver1
url = http://*.11:18088/driver

[driver2]
name = driver2
url = http://*.12:18088/driver

[driver3]
name = driver3
url = http://*.13:18088/driver

[driver4]
name = driver4
url = http://*.14:18088/driver

[driver5]
name = driver5
url = http://*.15:18088/driver

[driver6]
name = driver6
url = http://*.16:18088/driver

[driver7]
name = driver7
url = http://*.17:18088/driver

driver配置
在各机器的 /install-folder/conf/中找到drvier.conf，并配置内容如下：

[driver]
name=hostip:18088
url=http://hostip:18088/driver

###2.4 进程启动
在host1中，启动脚本start-all.sh 在host2~7中，启动脚本start-driver.sh

###2.5 界面展示
输入图片说明

##3 Cosbench使用 Cosbench的使用核心在于测试的配置文件编写，工具根据配置文件对整体集群进行读写测试。此处上一个配置文件示例：

<?xml version="1.0" encoding="UTF-8" ?>
<workload name="s3-sample" description="sample benchmark for s3">

  <storage type="s3" config="accesskey= ;secretkey= ;proxyhost=;proxyport=;endpoint=" />

  <workflow>

    <workstage name="init">
      <work type="init" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
    </workstage>

    <workstage name="prepare">
      <work type="prepare" workers="7" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,10);sizes=c(500)KB" />
    </workstage>

    <workstage name="main">
      <work name="main" workers="8" runtime="30">
     <!--   <operation type="read" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(1,10)" />-->
        <operation type="write" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(500)KB" />  
      </work>
    </workstage>

    <workstage name="cleanup">
      <work type="cleanup" workers="1" config="cprefix=s3testqwer;containers=r(1,2);objects=r(1,20)" />
    </workstage>

    <workstage name="dispose">
      <work type="dispose" workers="1" config="cprefix=s3testqwer;containers=r(1,2)" />
    </workstage>

  </workflow>

</workload>

###3.1 写测试此处我针对写测试进行一个简单的说明在main阶段配置wokrer 数量，控制输入压力， runtime控制程序运行时间，在operation的SIZE处设置文件大小。

<workstage name="main">
      <work name="main" workers="200" runtime="100">
        <operation type="write" ratio="100" config="cprefix=s3testqwer;containers=u(1,2);objects=u(11,20);sizes=c(500)KB" />  
      </work>
    </workstage>

3.2 写结果

输入图片说明

附

在配置文件的endpoint属性中配置时，请写Http://ip:port的方式，不过不标明http，他会默认采用https的方式，我之前没有配置Http时，日志报错如下：

2016-02-24 11:51:34,005 [INFO] [Log4jLogManager] - will append log to file /home/xuwenping/cosbench/cosbench/0.4.2.c3/log/mission/M2116744222.log
2016-02-24 11:51:34,225 [INFO] [NoneStorage] - performing PUT at /s3testqwer1
2016-02-24 11:51:38,436 [WARN] [S3Storage] - below exception encountered when creating bucket s3testqwer1: Unable to execute HTTP request: Connection to https://ip refused
2016-02-24 11:51:38,442 [INFO] [NoneStorage] - performing PUT at /s3testqwer2
2016-02-24 11:51:42,650 [WARN] [S3Storage] - below exception encountered when creating bucket s3testqwer2: Unable to execute HTTP request: Connection to https://ip refused

日志中写道，连接https://...... 时被拒绝。

转载于:https://my.oschina.net/myspaceNUAA/blog/620161