使用YCSB压测Cassandra 3.7

标签: cassandra ycsb cassandra压测
2025人阅读 评论(0) 收藏 举报
分类:
使用YCSB压测Cassandra 3.7
1、在https://github.com/brianfrankcooper/YCSB/获取源代码(Git或直接下载,我用Git下载到82%时候不动了,于是直接下载zip文件)


2、编译源代码
整个项目编译:mvn clean package
Cassandra单个编译:mvn -pl com.yahoo.ycsb:cassandra-binding -am clean package
我选择的是单个编译,比较快


3、编译成功后


显示Cassandra 2.1+ DB Binding .... SUCCESS就说明cassandra编译成功了  其他的不用管,直接去拿/home/yc/YCSB-master/cassandra/target/ycsb-cassandra-binding-0.11.0-SNAPSHOT.tar.gz就行了

#mkidr /usr/local/ycsb/
#cp ycsb-cassandra-binding-0.11.0-SNAPSHOT.tar.gz /usr/local/ycsb/
#cd /usr/local/ycsb/
#tar -zxvf ycsb-cassandra-binding-0.11.0-SNAPSHOT.tar.gz
#cd ycsb-cassandra-binding-0.11.0


4、用cassandra的cqlsh创建keyspace和cloumn family

(1)新建keyspace:

create keyspace usertable with replication = {'class':'SimpleStrategy', 'replication_factor':1};


(2)新建table也即cloumn family:

create table usertable (y_id varchar primary key,field0 varchar,field1 varchar,field2 varchar,field3 varchar,field4 varchar,field5 varchar,field6 varchar,field7 varchar,field8 varchar,field9 varchar);


5、查看ycsb命令格式

#cd bin
#ycsb
usage: ./ycsb command database [options]


Commands:
    load           Execute the load phase
    run            Execute the transaction phase
    shell          Interactive mode


Databases:
    accumulo       https://github.com/brianfrankcooper/YCSB/tree/master/accumulo
    aerospike      https://github.com/brianfrankcooper/YCSB/tree/master/aerospike
    arangodb       https://github.com/brianfrankcooper/YCSB/tree/master/arangodb
    asynchbase     https://github.com/brianfrankcooper/YCSB/tree/master/asynchbase
    basic          https://github.com/brianfrankcooper/YCSB/tree/master/basic
    cassandra-cql  https://github.com/brianfrankcooper/YCSB/tree/master/cassandra
    cassandra2-cql https://github.com/brianfrankcooper/YCSB/tree/master/cassandra2
    couchbase      https://github.com/brianfrankcooper/YCSB/tree/master/couchbase
    couchbase2     https://github.com/brianfrankcooper/YCSB/tree/master/couchbase2
    dynamodb       https://github.com/brianfrankcooper/YCSB/tree/master/dynamodb
    elasticsearch  https://github.com/brianfrankcooper/YCSB/tree/master/elasticsearch
    geode          https://github.com/brianfrankcooper/YCSB/tree/master/geode
    googlebigtable https://github.com/brianfrankcooper/YCSB/tree/master/googlebigtable
    googledatastore https://github.com/brianfrankcooper/YCSB/tree/master/googledatastore
    hbase094       https://github.com/brianfrankcooper/YCSB/tree/master/hbase094
    hbase098       https://github.com/brianfrankcooper/YCSB/tree/master/hbase098
    hbase10        https://github.com/brianfrankcooper/YCSB/tree/master/hbase10
    hypertable     https://github.com/brianfrankcooper/YCSB/tree/master/hypertable
    infinispan     https://github.com/brianfrankcooper/YCSB/tree/master/infinispan
    infinispan-cs  https://github.com/brianfrankcooper/YCSB/tree/master/infinispan
    jdbc           https://github.com/brianfrankcooper/YCSB/tree/master/jdbc
    kudu           https://github.com/brianfrankcooper/YCSB/tree/master/kudu
    mapkeeper      https://github.com/brianfrankcooper/YCSB/tree/master/mapkeeper
    memcached      https://github.com/brianfrankcooper/YCSB/tree/master/memcached
    mongodb        https://github.com/brianfrankcooper/YCSB/tree/master/mongodb
    mongodb-async  https://github.com/brianfrankcooper/YCSB/tree/master/mongodb
    nosqldb        https://github.com/brianfrankcooper/YCSB/tree/master/nosqldb
    orientdb       https://github.com/brianfrankcooper/YCSB/tree/master/orientdb
    rados          https://github.com/brianfrankcooper/YCSB/tree/master/rados
    redis          https://github.com/brianfrankcooper/YCSB/tree/master/redis
    riak           https://github.com/brianfrankcooper/YCSB/tree/master/riak
    s3             https://github.com/brianfrankcooper/YCSB/tree/master/s3
    solr           https://github.com/brianfrankcooper/YCSB/tree/master/solr
    tarantool      https://github.com/brianfrankcooper/YCSB/tree/master/tarantool
    voldemort      https://github.com/brianfrankcooper/YCSB/tree/master/voldemort


Options:
    -P file        Specify workload file
    -cp path       Additional Java classpath entries
    -jvm-args args Additional arguments to the JVM
    -p key=value   Override workload property
    -s             Print status to stderr
    -target n      Target ops/sec (default: unthrottled)
    -threads n     Number of client threads (default: 1)


Workload Files:
    There are various predefined workloads under workloads/ directory.
    See https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties
    for the list of workload properties.
ycsb: error: too few arguments

从命令格式里可以看出   -P可以加载一些配置文件    -p可以以键值对的方式加载一些配置   -s每隔一段时间输出执行信息  -threads线程数

6、新建cassandra连接文件(里面的属性可以在源码/home/yc/YCSB-master/cassandra/src/main/java/com/yahoo/ycsb/db/CassandraCQLClient.java中查看)

#vim cassandra.properties


hosts = spark131,spark130,spark129   #host列表,用逗号,隔开
port = 9042
cassandra.keyspace = usertable    #测试表
cassandra.username = ershixiong   #cassandra用户名
cassandra.password = 111111		  #cassandra密码
cassandra.readconsistencylevel = ANY
cassandra.writeconsistencylevel = ANY
cassandra.maxconnections = 100
cassandra.connecttimeoutmillis = 1000000000
cassandra.readtimeoutmillis  = 1000000000



7、配置workload

#vim workloads/workloada
workload=com.yahoo.ycsb.workloads.CoreWorkload
readallfields=false
readproportion=0.5
updateproportion=0.5
scanproportion=0
insertproportion=0
requestdistribution=zipfian
fieldcount 表示每条数据中的字段数,默认为 10;
fieldlength 表示每个字段的值的长度,默认为 100;
readallfields 域用来标识是否读取所有的所有的字段,取值有 ture 或 false;
readproportion,
updateproportion,
scanproportion,
insertproportion 分别表示该 workload中读、更新、扫描和插入操作占总操作的百分比,这四个值的和为 1;
requestdistribution 表示数据的分布情况,当前支持 uniform,zipfian 和 latest,默认为 uniform;
maxscanlength 域主要为扫描操作定义,定义了最大扫描的记录数量,默认为 1000;
scanlengthdistribution 域也是为扫描操作定义的,为每次扫描的长度定义相应的分布,默认是 uniform;
insertorder 域主要分两种 ordered 和 hashed,默认为 hashed;
operationcount 总共的 operation 数量;
maxexecutiontime 为该 workload 定义了最长的执行时间,单位为 s。
AverageLatency(平均潜伏期)平均潜伏期(average latency):指当磁头移动到数据所在的磁道后,然后等待所要的数据块继续转动(半圈或多些、少些)到磁头下的时间,单位为毫秒(ms)。平均潜伏期是越小越好,潜伏期小代表硬盘的读取数据的等待时间短,这就等于具有更高的硬盘数据传输率。


8、

#cd ..
#./bin/ycsb load cassandra-cql -P workloads/workloada -P cassandra.properties -p columnfamily=usertable -s -threads 20 > ./write7read3-log.log



数据接口名为cassandra-cql
加载ycsb的配置:workloads/workloada
加载cassandra的配置:cassandra.properties
columnfamily名称为usertable
20个线程执行




查看评论

Convert an UNSIGNED byte to a JAVA type

In JAVA, a byte always considered as signed when converted to another type. We must mask the sign bi...
  • wxyxl
  • wxyxl
  • 2001-05-04 14:13:00
  • 677

利用ycsb测试cassandra性能

java 、maven、ycsb 的安装及配置见这篇博客: http://blog.csdn.net/hs794502825/article/details/17309845 本篇博客主要...
  • hs794502825
  • hs794502825
  • 2013-12-15 16:14:58
  • 6073

ycsb cassandra 安装测试

1. YCSB安装wget https://github.com/downloads/brianfrankcooper/YCSB/ycsb-0.1.4.tar.gztar zxvf ycsb-0.1....
  • qianlong4526888
  • qianlong4526888
  • 2016-12-19 20:12:09
  • 493

利用YCSB工具对mongodb进行压测

1. 安装Java 去oracle官网下载jdk-7u15-linux-x64.rpm (直接yum安装的jdk版本貌似有问题) rpm –ivh jdk-7u15-linux-x64.rpm 2....
  • cug_jiang126com
  • cug_jiang126com
  • 2016-05-26 19:45:59
  • 4479

YCSB性能测试工具使用

在网上查In-Memory NoSQL性能测试的资料时,偶然间发现了这个性能测试工具YCSB,全称为“Yahoo! Cloud Serving Benchmark”。它内置了对常见NoSQL数据库和数...
  • dc_726
  • dc_726
  • 2015-02-28 22:46:39
  • 19274

Cassandra学习笔记 --- cassandra-driver使用

Cassandra学习笔记 --- cassandra-driver使用
  • u012965373
  • u012965373
  • 2016-12-02 18:43:57
  • 1531

简单的cassandra安装和测试

简单的cassandra安装配置和使用
  • u012978731
  • u012978731
  • 2016-11-02 13:54:37
  • 2086

YCSB简介

YCSB 如何运行:https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload 推荐操作顺利:https://github...
  • huozhanfeng
  • huozhanfeng
  • 2014-08-11 19:49:41
  • 2004

利用ycsb测试redis性能

1、jdk的安装 在http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html上面 下载...
  • hs794502825
  • hs794502825
  • 2013-12-13 23:06:24
  • 9090

ycsb安装使用方法

ycsb安装使用方法: 下载tar包,用tar -xfvz ycsb-0.1.4.tar.gz 解压缩后,cd 到 ycsb/bin 目录,用./ycsb可以查看命令格式 Usage: ./ycs...
  • haha72897
  • haha72897
  • 2012-06-14 21:05:49
  • 4397
    个人资料
    持之以恒
    等级:
    访问量: 8万+
    积分: 1120
    排名: 4万+
    最新评论