这两天主要是在最新版本的YCSB的基础上用mapkeeperclient打包并连接mapkeeper端的server。不得不说ycsb在创建mapkeeper client的时候就是有问题的。我踩了很多坑,我要记下来做笔记。
首先是下载最新的YCSB:
git clone https://github.com/brianfrankcooper/YCSB.git
然后执行下面命令:
cd YCSB
vim pom.xml
在pom.xml文件中把有关mapkeeper的部分取消注释:
取消注释部分:
<mapkeeper.version>1.0</mapkeeper.version>
<module>mapkeeper</module>
在mapkeeper目录下的pom.xml文件中添加:
<dependency>
<groupId>org.apache.thrift</groupId>
<artifactId>libthrift</artifactId>
<version>0.6.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>1.7.25</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<version>1.7.25</version>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.25</version>
</dependency>
敲黑板的重点是这里thrift一定要是0.6.1版本的!!!我试过最新0.12.0还有0.8.0都不行。这才是我这几天一直mvn打包的时候报错的原因,如果不用这个会报以下错误:
Exception in thread "Thread-2" java.lang.IncompatibleClassChangeError: Implementing class
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at com.yahoo.ycsb.db.MapKeeperClient.init(MapKeeperClient.java:69)
at com.yahoo.ycsb.DBWrapper.init(DBWrapper.java:86)
at com.yahoo.ycsb.ClientThread.run(ClientThread.java:91)
at java.lang.Thread.run(Thread.java:748)
我在网上搜索了好久也不知道上面是啥错误,原来是mapkeeper在mvn打包的时候只支持0.6.1版本。然后就是对于MapKeeperClient的修改了,在这里最新版的MapKeeperClient.java文件中是有一些问题的就是把一些本该是用com.yahoo.ycsb.Status类型的表示用常用的int表示了。所以会报checkstyle的错误。同时格式也会不对显示类似于本该空着几格实际空了几格的报错,因为可能是tab问题。所以需要手动修改。修改如下完整的mapkeeperclient文件:
/**
* Copyright (c) 2012 YCSB contributors. All rights reserved.
* <p>
* Licensed under the Apache License, Version 2.0 (the "License"); you
* may not use this file except in compliance with the License. You
* may obtain a copy of the License at
* <p>
* http://www.apache.org/licenses/LICENSE-2.0
* <p>
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
* implied. See the License for the specific language governing
* permissions and limitations under the License. See accompanying
* LICENSE file.
*/
package com.yahoo.ycsb.db;
//import java.nio.CharBuffer;
//import java.util.Arrays;
//import java.nio.charset.Charset;
//import java.nio.charset.CharsetDecoder;
import java.nio.ByteBuffer;
import java.util.HashMap;
import java.util.Properties;
import java.util.Set;
import java.util.Vector;
import java.util.Map;
import org.apache.thrift.TException;
import org.apache.thrift.protocol.TBinaryProtocol;
import org.apache.thrift.protocol.TProtocol;
import org.apache.thrift.transport.TFramedTransport;
import org.apache.thrift.transport.TSocket;
import org.apache.thrift.transport.TTransport;
//import org.apache.commons.codec.binary.Base64;
import com.yahoo.mapkeeper.BinaryResponse;
import com.yahoo.mapkeeper.MapKeeper;
import com.yahoo.mapkeeper.Record;
import com.yahoo.mapkeeper.RecordListResponse;
import com.yahoo.mapkeeper.ResponseCode;
import com.yahoo.mapkeeper.ScanOrder;
import com.yahoo.ycsb.ByteIterator;
import com.yahoo.ycsb.DB;
import com.yahoo.ycsb.StringByteIterator;
import com.yahoo.ycsb.workloads.CoreWorkload;
import com.yahoo.ycsb.Status;
/**
* YCSB binding for <a href="http://redis.io/">Redis</a>.
*
* See {@code redis/README.md} for details.
*/
public class MapKeeperClient extends DB {