女主宣言
在对象存储迭代研发、测试过程中,为了方便、准确验证 S3 接口协议兼容性,本文作者对Ceph官方采用的兼容性测试工具s3-tests进行了调研,并对其配置以及使用做出了详细介绍,相信对于s3的使用者,会起到实质性的帮助,下来就跟随作者一探究竟吧。
PS:丰富的一线技术、多元化的表现形式,尽在“360云计算”,点关注哦!
1
简介
在对象存储迭代研发、测试过程中,为了方便、准确验证 S3 接口协议兼容性,我们内部配置了 Ceph 官方采用的兼容性测试工具 s3-tests ,该工具基于 Python Nose 测试框架和官方的 Boto 库(同时支持 boto 2 / boto 3),并通过结合 HTML 扩展可将测试结果输出到Web界面,通过页面查询相关测试统计、分析结果。
s3-tests
S3 compatibility tests - This is a set of unofficial Amazon AWS S3 compatibility tests, that can be useful to people implementing software that exposes an S3-like API. The tests use the Boto2 and Boto3 libraries.
仓库地址:https://github.com/ceph/s3-tests
Nose 框架
Nose 是一种流行的 Python 单元测试框架扩展,它可以方便地自动运行一批测试和插件,比如度量代码覆盖率。Nose 能支持基于函数的测试,也能支持与 unittest 相类似的基于测试类的测试方式。
nose-html-reporting 扩展
Nose plugin that generates a nice html test report with ability of using template based on jinja2 templates from any folder.
源自: https://github.com/ionelmc/nose-htmloutput ,增加了对 jinja2 模板支持。
2
配置 s3-tests 实践
下载源码
# git clone https://github.com/ceph/s3-tests.git
安装、部署
# sudo apt-get install python-virtualenv # cd s3-tests
启动测试运行环境,安装依赖软件:
# ./bootstrap
创建配置文件
首先,需要通过 S3 管理接口提前创建好配置文件中的用户、access_key,secret_key 等必要配置信息。
创建 config.yaml 增加如下内容:
[DEFAULT] ## this p is just used as default for all the "s3 *" ## ps, you can place these variables also directly there ## replace with e.g. "localhost" to run against local software host = test-shyc2.s3.360.local # 也可用本机 S3 Gateway IP 地址 ## uncomment the port to use something other than 80 port = 80 ## say "no" to disable TLS is_secure = no [fixtures] ## all the buckets created will start with this prefix; ## {random} will be filled with random characters to pad ## the prefix to 30 characters long, and avoid collisions bucket prefix = s3test-{random}- [s3 main] ## the tests assume two accounts are defined, "main" and "alt". ## user_id is a 64-character hexstring user_id = infra-s3 ## display name typically looks more like a unix login, "jdoe" etc display_name = infra-s3 ## replace these with your access keys access_key = infra-ak secret_key = infra-sk [s3 alt] ## another user account, used for ACL-related tests user_id = infra-alt display_name = infra-alt ## the "alt" user needs to have email set, too email = infra-alt@qihoo.net access_key = infra-alt-ak secret_key = infra-alt-sk
查看测试项
~/s3-tests# S3TEST_CONF=./config.yaml ./virtualenv/bin/nosetests -v --collect-only s3tests.functional.test_headers.test_object_create_bad_md5_invalid_short ... ok s3tests.functional.test_headers.test_object_create_bad_md5_bad ... ok s3tests.functional.test_headers.test_object_create_bad_md5_empty ... ok s3tests.functional.test_headers.test_object_create_bad_md5_unreadable ... ok s3tests.functional.test_headers.test_object_create_bad_md5_none ... ok s3tests.functional.test_headers.test_object_create_bad_expect_mismatch ... ok s3tests.functional.test_headers.test_object_create_bad_expect_empty ... ok s3tests.functional.test_headers.test_object_create_bad_expect_none ... ok s3tests.functional.test_headers.test_object_create_bad_expect_unreadable ... ok s3tests.functional.test_headers.test_object_create_bad_contentlength_empty ... ok s3tests.functional.test_headers.test_object_create_bad_contentlength_negative ... ok s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok s3tests.functional.test_headers.test_object_create_bad_contentlength_unreadable ... ok s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_above ... ok s3tests.functional.test_headers.test_object_create_bad_contenttype_invalid ... ok s3tests.functional.test_headers.test_object_create_bad_contenttype_empty ... ok s3tests.functional.test_headers.test_object_create_bad_contenttype_none ... ok s3tests.functional.test_headers.test_object_create_bad_contenttype_unreadable ... ok s3tests.functional.test_headers.test_object_create_bad_authorization_unreadable ... ok s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok
3
Nose Html 扩展
配置静态资源服务器
搭建一个 HTTP 资源服务器用于展示 nose 输出。
在线安装 nginx / httpd 即可,下文中直接将输出文件保存到默认路径 /var/www/html/ 下。
安装 nose-html-reporting
官方地址 : https://pypi.org/project/nose-html-reporting/#description。
安装 nose-html-reporting。
# ./virtualenv/bin/pip install nose-html-reporting
验证,有如下输出,代表扩展已安装,支持输出 html 文件。
~/s3-tests# ./virtualenv/bin/nosetests -h | grep html
--with-html Enable plugin HtmlReport: Output test results as
pretty html. [NOSE_WITH_HTML]
--html-report=FILE Path to html file to store the report in. Default is
nosetests.html in the working directory
--html-report-template=FILE
Path to html template file in with jinja2
format.Default is report.html in the lib
--cover-html Produce HTML coverage information
--cover-html-dir=DIR Produce HTML coverage information in dir
~/s3-tests#
执行单个用例
单独运行 test_bucket_list_empty 用例,指定 html 模板文件为 report2.jinja2。
执行如下命令:
~/s3-tests# S3TEST_CONF=./config.yaml ./virtualenv/bin/nosetests s3tests.functional.test_s3:test_bucket_list_empty -v --with-html --html-report=/var/www/html/index.html --html-report-template=virtualenv/lib/python2.7/site-packages/nose_html_reporting/templates/report2.jinja2。
示例:
~/s3-tests# S3TEST_CONF=./config.yaml ./virtualenv/bin/nosetests s3tests.functional.test_s3:test_bucket_list_empty -v --with-html --html-report=/var/www/html/index.html --html-report-template=virtualenv/lib/python2.7/site-packages/nose_html_reporting/templates/report2.jinja2
s3tests.functional.test_s3.test_bucket_list_empty ... ok
----------------------------------------------------------------------
HTML: /var/www/html/index.html
----------------------------------------------------------------------
Ran 1 test in 0.322s
OK
~/s3-tests#
打开 web 端,查看测试结果:
部分测试
运行 atomic read/write 单元测试用例:
#!/bin/bash
cases="\
s3tests.functional.test_s3:test_atomic_read_1mb \
s3tests.functional.test_s3:test_atomic_read_4mb \
s3tests.functional.test_s3:test_atomic_read_8mb \
s3tests.functional.test_s3:test_atomic_write_1mb \
s3tests.functional.test_s3:test_atomic_write_4mb \
s3tests.functional.test_s3:test_atomic_write_8mb \
s3tests.functional.test_s3:test_atomic_dual_write_1mb \
s3tests.functional.test_s3:test_atomic_dual_write_4mb \
s3tests.functional.test_s3:test_atomic_dual_write_8mb \
s3tests.functional.test_s3:test_atomic_conditional_write_1mb"
S3_USE_SIGV4=1 S3TEST_CONF=config.yaml virtualenv/bin/nosetests $cases -a \
auth_aws4 -x --debug-log=./tests.log --cover-html --process-timeout=60 \
--processes=10
如上通过指定特定的 case 进行测试,配置 ENV 中 S3_USE_SIGV4=1 和参数 -a auth_aws4 使能 AWS V4 鉴权(默认为 V2), --debug-log 用于记录测试过程中详细日志。
执行全量测试
执行如下命令:
S3_USE_SIGV4=1 S3TEST_CONF=./config.yaml ./virtualenv/bin/nosetests -a auth_aws4 -v --with-html --html-report=/var/www/html/index.html
示例输出:
s3tests.functional.test_headers.test_object_create_bad_md5_invalid_short ... ok
s3tests.functional.test_headers.test_object_create_bad_md5_bad ... ok
s3tests.functional.test_headers.test_object_create_bad_md5_empty ... ok
s3tests.functional.test_headers.test_object_create_bad_md5_unreadable ... ok
s3tests.functional.test_headers.test_object_create_bad_md5_none ... ok
s3tests.functional.test_headers.test_object_create_bad_expect_mismatch ... ok
s3tests.functional.test_headers.test_object_create_bad_expect_empty ... ok
s3tests.functional.test_headers.test_object_create_bad_expect_none ... ok
s3tests.functional.test_headers.test_object_create_bad_expect_unreadable ... ok
s3tests.functional.test_headers.test_object_create_bad_contentlength_empty ... ok
s3tests.functional.test_headers.test_object_create_bad_contentlength_negative ... ok
s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok
s3tests.functional.test_headers.test_object_create_bad_contentlength_unreadable ... ok
s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_above ... FAIL
s3tests.functional.test_headers.test_object_create_bad_contenttype_invalid ... ok
s3tests.functional.test_headers.test_object_create_bad_contenttype_empty ... ok
s3tests.functional.test_headers.test_object_create_bad_contenttype_none ... ok
s3tests.functional.test_headers.test_object_create_bad_contenttype_unreadable ... FAIL
s3tests.functional.test_headers.test_object_create_bad_authorization_unreadable ... FAIL
s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok
s3tests.functional.test_headers.test_object_create_date_and_amz_date ... ok
s3tests.functional.test_headers.test_object_create_amz_date_and_no_date ... ok
最后见到如下统计输出,并且可以看到 HTML 输出到 /var/www/html/index.html 中了,代表任务成功运行,现在可以通过界面查看 HTML 端统计结果。
----------------------------------------------------------------------
HTML: /var/www/html/index.html
----------------------------------------------------------------------
Ran 570 tests in 542.311s
FAILED (SKIP=85, errors=26, failures=22)
4
统计分析
概览统计
查询测试结果,点击 Detail 展开单元测试项运行状态。
详细信息
通过点击失败用例对应的 Fail 按钮,查询具体错误信息。
通常我们根据断言信息及请求、响应 header 就可以定位到具体问题了。
5
自定义测试项
对于我们自定义的类S3接口(如自定义Meta信息、私有管理接口等),也可以很方便的通过扩展现有单元测试函数来实现,下面是 HTTP Range 请求下载功能的测试函数,函数位置:s3-tests/s3tests/functional/test_s3.py 中 Http Range 函数。
示例:
@attr(resource='object')
@attr(method='get')
@attr(operation='range')
@attr(assertion='returns correct data, 206')
def test_ranged_big_request_response_code():
bucket = get_new_bucket()
key = bucket.new_key('testobj')
string = os.urandom(8 * 1024 * 1024)
# 上传对象
key.set_contents_from_string(string)
# 设置 header,指定 Range 数据段
key.open('r', headers={'Range': 'bytes=3145728-5242880'})
status = key.resp.status
content_range = key.resp.getheader('Content-Range')
fetched_content = ''
for data in key:
fetched_content += data;
key.close()
# 验证 Gateway 响应状态码,数据长度等
eq(fetched_content, string[3145728:5242881])
eq(status, 206)
eq(content_range, 'bytes 3145728-5242880/8388608')
可以此为例,进行自定义用例扩展。
360云计算
由360云平台团队打造的技术分享公众号,内容涉及数据库、大数据、微服务、容器、AIOps、IoT等众多技术领域,通过夯实的技术积累和丰富的一线实战经验,为你带来最有料的技术分享