HBase实现谷粒微博案例
前言
最近刚刚在b站上看完了尚硅谷hbase相关课程，在这里记录一下，完成这个项目遇到的坑，以及案例结果(而且不要吐槽的英语注释，我是个英语渣，大家尽量意会一下…)。如果博客有什么不对的地方，欢迎大佬指正。
一、启动集群
由于HBase是基于hadoop的一个非结构化数据库，所以需要启动hadoop集群，并且还需要启动我们的“润滑器” zookeeper。
我使用的版本是
CentOS 6
Hadoop 3
HBase 3
Zookeeper 3 （每一个虚拟机都要启动zookeeper）
虚拟机一共三台，分别是hadoop102，hadoop103，hadoop104
编译器使用的是IDEA。
二、功能实现
1.创建工程
这里我只是给出代码以及一些相关解释，具体的需求分析以及更加详细的解释，可以看后面的参考链接，我将会给出b站尚硅谷视频的链接，有兴趣的可以去看看。
打开IDEA，创建一个新的maven工程，然后进行pom.xml格式如下。这里是需要导入相关的依赖包，以及各种依赖的版本，build一下就会开始进行下载。
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>HBase</artifactId>
<groupId>org.example</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>guli-weibo</artifactId>
<dependencies>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-server</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.3.1</version>
</dependency>
</dependencies>
<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
</project>
然后在工程中创建5个文件夹（其中一个是空的，也可以不创建），后续需要添加代码。如图：
ok，现在我们就可以快乐的编写代码了。
2.constants包
在这里我们定义的是一些常量，例如是命名空间、表明、列名等，以便后面我们需要使用的时候反复重写，防止写错，也可以进行一个解耦合。
package constants;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.*;
public class Constants {
public static Configuration CONFIGURATION = null;
// HBase setting 这里不知道为什么要这要写才可以，视频里面没有写这个也可以运行，
// 但是我就会报错，说无法连接，希望有大佬可以解释一下。
// 那个set里面第二个放的是你们集群的名称，可能会不一样，看你们自己的命名了。
static {
CONFIGURATION = HBaseConfiguration.create();
CONFIGURATION.set("hbase.zookeeper.quorum", "hadoop102,hadoop103,hadoop104");
}
// HBase namespace 这个是命名空间
public static final String NAMESPACE = "weibo";
// weibo content table 这个是内容表
public static final String CONTENT_TABLE = "weibo:content";
public static String CONTENT_TABLE_CF = "info";
public static int CONTENT_TABLE_VERSIONS = 1;
// user relationship table 这个是关系表
public static final String RELATION_TABLE = "weibo:relation";
public static final String RELATION_TABLE_CF1 = "attends";
public static final String RELATION_TABLE_CF2 = "fans";
public static final int RELATION_TABLE_VERSIONS = 1;
// receipt email box 这个是收件表
public static final String INBOX_TABLE = "weibo:inbox";
public static final String INBOX_TABLE_CF = "info";
public static final int INBOX_TABLE_VERSIONS = 2;
}
3.utils包
这里主要是进行DDL的操作，例如创建命名空间，以及对表之类的操作。
代码总览，下面再分段解释：
package utils;
import constants.Constants;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.NamespaceDescriptor;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.Admin;
import org.apache.hadoop.hbase.client.Connection;
import org.apache.hadoop.hbase.client.ConnectionFactory;
import org.apache.hadoop.hbase.client.Table;
import java.io.IOException;
/**
* 1 create namespace
* 2 judge table exist or not
* 3 create three tables
* 4
*/
public class HBaseUtil {
// 1 create namespace
public static void createNameSpace(String namespace) throws IOException {
// 1 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get the admin object
Admin admin = connection.getAdmin();
// 1 create namespace description
NamespaceDescriptor build = NamespaceDescriptor.create(namespace).build();
// 2 create namespace
admin.createNamespace(build);
// 3 close resource
admin.close();
connection.close();
}
// 2 judge the table exist or not
private static boolean isTableExist(String tableName) throws IOException {
// 1 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get the admin object
Admin admin = connection.getAdmin();
// 3 judge the table exist or not
boolean result = admin.tableExists(TableName.valueOf(tableName));
// 4 close resource
admin.close();
connection.close();
// return value
return result;
}
// 3 create table
public static void createTable(String tableName, int versions, String... cfs) throws IOException {
// 1 judge the command value
if (cfs.length <= 0) {
System.out.println("column information is not available.");
return ;
}
// 2 judge the table exist or not
if (isTableExist(tableName)) {
System.out.println(tableName + "table is exist.");
return;
}
// 3 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 4 get the admin object
Admin admin = connection.getAdmin();
// 5 create table description
HTableDescriptor hTableDescriptor = new HTableDescriptor(TableName.valueOf(tableName));
// 6 cycle add column information
for (String cf : cfs) {
HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(cf);
// 7 set the version
hColumnDescriptor.setMaxVersions(versions);
hTableDescriptor.addFamily(hColumnDescriptor);
}
// 8 create table
admin.createTable(hTableDescriptor);
// close resource
admin.close();
connection.close();
}
}
3.1 createNameSpace 创建命名空间
// 1 create namespace
public static void createNameSpace(String namespace) throws IOException {
// 1 get the connection object 创建一个连接的对象
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get the admin object 创建一个admin的对象，
// 对hbase操作基本上通admin来操作
Admin admin = connection.getAdmin();
// 1 create namespace description 创建一个命名空间的解释器
NamespaceDescriptor build = NamespaceDescriptor.create(namespace).build();
// 2 create namespace 创建命名空间
admin.createNamespace(build);
// 3 close resource 关闭资源
admin.close();
connection.close();
}
3.2 isTableExist 判断表是否存在
// 2 judge the table exist or not
private static boolean isTableExist(String tableName) throws IOException {
// 1 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get the admin object
Admin admin = connection.getAdmin();
// 3 judge the table exist or not.
// tableExists这个函数是需要传入TableName参数的，所以需要转换。
boolean result = admin.tableExists(TableName.valueOf(tableName));
// 4 close resource
admin.close();
connection.close();
// return value
return result;
}
3.3 createTable 创建表
这一块蛮重要的，但是也是固定的套路。
// 3 create table
public static void createTable(String tableName, int versions, String... cfs) throws IOException {
// 1 judge the command value 判断一下传入的参数是否小于等于0
if (cfs.length <= 0) {
System.out.println("column information is not available.");
return ;
}
// 2 judge the table exist or not 判断表是否存在
if (isTableExist(tableName)) {
System.out.println(tableName + "table is exist.");
return;
}
// 3 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 4 get the admin object
Admin admin = connection.getAdmin();
// 5 create table description
// 创建表描述器，对表的操作都是通过描述器来操作。
HTableDescriptor hTableDescriptor = new HTableDescriptor(TableName.valueOf(tableName));
// 6 cycle add column information
for (String cf : cfs) {
// 这里是创建一个列描述器
HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(cf);
// 7 set the version
hColumnDescriptor.setMaxVersions(versions);
// 将列描述器加入到表描述器当中，也就是把列族加进去。
hTableDescriptor.addFamily(hColumnDescriptor);
}
// 8 create table
admin.createTable(hTableDescriptor);
// close resource
admin.close();
connection.close();
}
4.dao包
这里是一些DML的操作，例如一些在表中插入数据，或者是删除数据的操作。
老规矩，先来一个代码总览(比较长…)：
package dao;
import constants.Constants;
import org.apache.hadoop.hbase.Cell;
import org.apache.hadoop.hbase.CellUtil;
import org.apache.hadoop.hbase.TableName;
import org.apache.hadoop.hbase.client.*;
import org.apache.hadoop.hbase.filter.CompareFilter;
import org.apache.hadoop.hbase.filter.RowFilter;
import org.apache.hadoop.hbase.filter.SubstringComparator;
import org.apache.hadoop.hbase.util.Bytes;
import java.io.IOException;
import java.util.ArrayList;
/**
* 1 send the weibo
* 2 delete the weibo
* 3 focus the user
* 4 check off user
* 5 get the user's weibo detail
* 6 get the user's initialize page
*/
public class HBaseDao {
// 1 send the weibo
public static void publishWeiBo(String uid, String content) throws IOException {
// 1 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get the admin object
Admin admin = connection.getAdmin();
// 1 operate weibo content table
// 1.1 get the weibo content table object
Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));
// 1.2 get the current time
long ts = System.currentTimeMillis();
// 1.3 get the Rowkey
String rowKey = uid + "_" + ts;
// 1.4 create put object
Put conPut = new Put(Bytes.toBytes(rowKey));
// 1.5 Assignment the put object
conPut.addColumn(Bytes.toBytes(Constants.CONTENT_TABLE_CF), Bytes.toBytes("content"), Bytes.toBytes(content));
// 1.6 execute insert data operation
conTable.put(conPut);
// 2 operate weibo email box
// 2.1 get user relation table object
Table relaTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));
// 2.2 get the fans's information of the person who publish weibo in current
Get get = new Get(Bytes.toBytes(uid));
get.addFamily(Bytes.toBytes(Constants.RELATION_TABLE_CF2));
Result result = relaTable.get(get);
// 2.3 create a map, in order to store the put object of weibo content table
ArrayList<Put> inboxPuts = new ArrayList<>();
// 2.4 traverse the fans list
for (Cell cell : result.rawCells()) {
// 2.5 create the put object of weibo infobox table
Put inboxPut = new Put(CellUtil.cloneQualifier(cell));
// 2.6 assignment the information to the inbox table
inboxPut.addColumn(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(uid), Bytes.toBytes(rowKey));
// 2.7 store the put object into the inbox table
inboxPuts.add(inboxPut);
}
// 2.8 judge fans exist or not
if (inboxPuts.size() > 0) {
// get the inbox object
Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));
// execute the insert operation
inboxTable.put(inboxPuts);
// close the inbox table
inboxTable.close();
}
// close the resource
relaTable.close();
conTable.close();
connection.close();
}
// 2 focus the user
public static void addAttends(String uid, String... attends) throws IOException {
// judge if or not exist attends
if (attends.length <= 0) {
System.out.println("please choose attention person");
return ;
}
// get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// first, operate user relationship table
// 1 get the user relationship table object
Table relaTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));
// 2 create a map, in order to store a put object of user relationship table
ArrayList<Put> relaPuts = new ArrayList<>();
// 3 create an operator put object
Put uidPut = new Put(Bytes.toBytes(uid));
// 4 cycle create a put object of attended person
for (String attend : attends) {
// 5 assignment value to operator's put object
uidPut.addColumn(Bytes.toBytes(Constants.RELATION_TABLE_CF1), Bytes.toBytes(attend), Bytes.toBytes(attend));
// 6 create a put object of attended person
Put attendPut = new Put(Bytes.toBytes(attend));
// 7 assignment value to attended person's put object
attendPut.addColumn(Bytes.toBytes(Constants.RELATION_TABLE_CF2), Bytes.toBytes(uid), Bytes.toBytes(uid));
// 8 put the attended person's put object into map
relaPuts.add(attendPut);
}
// 9 add operator's put object into map
relaPuts.add(uidPut);
// 10 execute user relationship table the insertion operation
relaTable.put(relaPuts);
// second operate inbox table
// 1 get the weibo content table object
Table contTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));
// 2 create put object of inbox table
Put inboxPut = new Put(Bytes.toBytes(uid));
// 3 cycle attends, get the every attended person current published weibo
for (String attend : attends) {
// 4 get the attended person's current published weibo -> map resultScanner
Scan scan = new Scan(Bytes.toBytes(attend + "_"), Bytes.toBytes(attend + "|"));
ResultScanner resultScanner = contTable.getScanner(scan);
// define a time stamp
long ts = System.currentTimeMillis();
// 5 traverse the value
for (Result result : resultScanner) {
// 6 assignment the value into the inbox table
inboxPut.addColumn(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(attend), ts++, result.getRow());
}
}
// judge the put object exist or not
if (!inboxPut.isEmpty()) {
// get the inbox table object
Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));
// insert data
inboxTable.put(inboxPut);
// close inboxTable connection
inboxTable.close();
}
// close resource
relaTable.close();
contTable.close();
connection.close();
}
// 3 delete attention
public static void deleteAttends(String uid, String... dels) throws IOException {
if (dels.length <= 0) {
System.out.println("please choose deletion person");
return;
}
// get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// first operate user relationship table
// 1 get the user relationship table object
Table relatTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));
// 2 create a map, in order to store the delete object of user relationship table
ArrayList<Delete> relatDelete = new ArrayList<>();
// 3 create the delete object of operator
Delete uidDelete = new Delete(Bytes.toBytes(uid));
// 4 cycle create the delete object of delete attention
for (String del : dels) {
// 5 assignment value to operator's delete object
uidDelete.addColumns(Bytes.toBytes(Constants.RELATION_TABLE_CF1), Bytes.toBytes(del));
// 6 create the conceal person's(取关者...) deletion object
Delete delDelte = new Delete(Bytes.toBytes(del));
// 7 assignment value to the conceal person's deletion object
delDelte.addColumns(Bytes.toBytes(Constants.RELATION_TABLE_CF2), Bytes.toBytes(uid));
// 8 add the conceal person's deletion object into the map
relatDelete.add(delDelte);
}
// 9 add the operator's deletion object into the map
relatDelete.add(uidDelete);
// 10 delete the data in user relationship table
relatTable.delete(relatDelete);
// second operate inbox table
// 1 get the inbox table object
Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));
// 2 create the deletion object of operator
Delete inboxDelete = new Delete(Bytes.toBytes(uid));
// 3 assignment value to operator's deletion object
for (String del : dels) {
inboxDelete.addColumns(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(del));
}
// 4 delete data in inbox table
inboxTable.delete(inboxDelete);
// close resource
relatTable.close();
inboxTable.close();
connection.close();
}
// 4 get the initial page data
public static void getInit(String uid) throws IOException {
// 1 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get inbox table object
Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));
// 3 get the content table object
Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));
// 4 create inbox get object, and get the data max version
Get inboxGet = new Get(Bytes.toBytes(uid));
inboxGet.setMaxVersions();
Result result = inboxTable.get(inboxGet);
// 5 traverse the data
for (Cell cell : result.rawCells()) {
// 6 construct the content table get object
Get conGet = new Get(CellUtil.cloneValue(cell));
// 7 get the object data
Result conResult = conTable.get(conGet);
// 8 print data
for (Cell conCell : conResult.rawCells()) {
System.out.println("RK:" + Bytes.toString(CellUtil.cloneRow(conCell)) +
", CF:" + Bytes.toString(CellUtil.cloneFamily(conCell)) +
", CN:" + Bytes.toString(CellUtil.cloneQualifier(conCell)) +
", Value:" + Bytes.toString(CellUtil.cloneValue(conCell)));
}
}
// 9 close resource
inboxTable.close();
conTable.close();
connection.close();
}
// 5 get the all the content of somebody weibo
public static void getWeibo(String uid) throws IOException {
// 1 get the connection
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get the weibo content table
Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));
// construct scan object
Scan scan = new Scan();
// construct filter
RowFilter rowFilter = new RowFilter(CompareFilter.CompareOp.EQUAL, new SubstringComparator(uid + "_"));
scan.setFilter(rowFilter);
// 4 get the data
ResultScanner resultScanner = conTable.getScanner(scan);
// 5 printf data
for (Result result : resultScanner) {
for (Cell cell : result.rawCells()) {
System.out.println("RK:" + Bytes.toString(CellUtil.cloneRow(cell)) +
", CF:" + Bytes.toString(CellUtil.cloneFamily(cell)) +
", CN:" + Bytes.toString(CellUtil.cloneQualifier(cell)) +
", Value:" + Bytes.toString(CellUtil.cloneValue(cell)));
}
}
// 6 close resource
conTable.close();
connection.close();
}
}
4.1 发微博功能
这一个功能比较难，因为不仅对于当前发布微博的人的内容表需要进行更新，并且需要对当前这个人的粉丝的收件表也需要更新，因为粉丝需要看到关注的人的最新动态。所以这里需要对两张表，多个人进行操作。
// 1 publish the weibo
public static void publishWeiBo(String uid, String content) throws IOException {
// 1 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get the admin object
Admin admin = connection.getAdmin();
// 1 operate weibo content table 对微博内容表进行操作
// 1.1 get the weibo content table object
Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));
// 1.2 get the current time
long ts = System.currentTimeMillis();
// 1.3 get the Rowkey
String rowKey = uid + "_" + ts;
// 1.4 create put object 将需要put的内容对象先创建好
Put conPut = new Put(Bytes.toBytes(rowKey));
// 1.5 Assignment the put object 给put对象赋值
conPut.addColumn(Bytes.toBytes(Constants.CONTENT_TABLE_CF), Bytes.toBytes("content"), Bytes.toBytes(content));
// 1.6 execute insert data operation
// 执行put操作，即对表进行插入数据
conTable.put(conPut);
// 2 operate weibo email box 操作微博收件表
// 2.1 get user relation table object
Table relaTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));
// 2.2 get the fans's information of the person who publish weibo in current
// 得到最近发布微博的被关注者的粉丝的信息
Get get = new Get(Bytes.toBytes(uid));
get.addFamily(Bytes.toBytes(Constants.RELATION_TABLE_CF2));
Result result = relaTable.get(get);
// 2.3 create a map, in order to store the put object of weibo content table
// 创建一个集合为了存储微博的内容
ArrayList<Put> inboxPuts = new ArrayList<>();
// 2.4 traverse the fans list
// 遍历uid的粉丝
for (Cell cell : result.rawCells()) {
// 2.5 create the put object of weibo infobox table
// 创建粉丝的单元格
Put inboxPut = new Put(CellUtil.cloneQualifier(cell));
// 2.6 assignment the information to the inbox table
// 对粉丝的收件表进行赋值，更新关注的人的最新微博
inboxPut.addColumn(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(uid), Bytes.toBytes(rowKey));
// 2.7 store the put object into the inbox table
inboxPuts.add(inboxPut);
}
// 2.8 judge fans exist or not
// 判断当前发布微博的人是否有粉丝
if (inboxPuts.size() > 0) {
// get the inbox object
Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));
// execute the insert operation
inboxTable.put(inboxPuts);
// close the inbox table
inboxTable.close();
}
// close the resource
relaTable.close();
conTable.close();
connection.close();
}
4.2 关注功能
关注某个人之后，需要获得这个人的最近发布的微博，以及在微博关系表当中需要添加新的关系。代码也比较长…
// 2 focus the user
public static void addAttends(String uid, String... attends) throws IOException {
// judge if or not exist attends
if (attends.length <= 0) {
System.out.println("please choose attention person");
return ;
}
// get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// first, operate user relationship table
// 操作微博关系表
// 1 get the user relationship table object
Table relaTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));
// 2 create a map, in order to store a put object of user relationship table
// 创建关注的集合，因为有可能一次性关注多个人。
ArrayList<Put> relaPuts = new ArrayList<>();
// 3 create an operator put object
Put uidPut = new Put(Bytes.toBytes(uid));
// 4 cycle create a put object of attended person
for (String attend : attends) {
// 5 assignment value to operator's put object
uidPut.addColumn(Bytes.toBytes(Constants.RELATION_TABLE_CF1), Bytes.toBytes(attend), Bytes.toBytes(attend));
// 6 create a put object of attended person
// 这是被关注者对象
Put attendPut = new Put(Bytes.toBytes(attend));
// 7 assignment value to attended person's put object
attendPut.addColumn(Bytes.toBytes(Constants.RELATION_TABLE_CF2), Bytes.toBytes(uid), Bytes.toBytes(uid));
// 8 put the attended person's put object into map
relaPuts.add(attendPut);
}
// 9 add operator's put object into map
relaPuts.add(uidPut);
// 10 add data into user relationship table
relaTable.put(relaPuts);
// second operate inbox table
// 1 get the weibo content table object
Table contTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));
// 2 create put object of inbox table
Put inboxPut = new Put(Bytes.toBytes(uid));
// 3 cycle attends, get the every attended person current published weibo
for (String attend : attends) {
// 4 get the attended person's current published weibo -> map resultScanner
// 获得被关注者最近发布的微博
Scan scan = new Scan(Bytes.toBytes(attend + "_"), Bytes.toBytes(attend + "|"));
ResultScanner resultScanner = contTable.getScanner(scan);
// define a time stamp
long ts = System.currentTimeMillis();
// 5 traverse the value
for (Result result : resultScanner) {
// 6 assignment the value into the inbox table
// 循环放入粉丝的收件表中
inboxPut.addColumn(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(attend), ts++, result.getRow());
}
}
// judge the put object exist or not
if (!inboxPut.isEmpty()) {
// get the inbox table object
Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));
// insert data
inboxTable.put(inboxPut);
// close inboxTable connection
inboxTable.close();
}
// close resource
relaTable.close();
contTable.close();
connection.close();
}
4.3 取消关注
// 3 delete attention
public static void deleteAttends(String uid, String... dels) throws IOException {
if (dels.length <= 0) {
System.out.println("please choose deletion person");
return;
}
// get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// first operate user relationship table
// 1 get the user relationship table object
Table relatTable = connection.getTable(TableName.valueOf(Constants.RELATION_TABLE));
// 2 create a map, in order to store the delete object of user relationship table
ArrayList<Delete> relatDelete = new ArrayList<>();
// 3 create the delete object of operator
Delete uidDelete = new Delete(Bytes.toBytes(uid));
// 4 cycle create the delete object of delete attention
for (String del : dels) {
// 5 assignment value to operator's delete object
uidDelete.addColumns(Bytes.toBytes(Constants.RELATION_TABLE_CF1), Bytes.toBytes(del));
// 6 create the conceal person's(取关者...) deletion object
Delete delDelte = new Delete(Bytes.toBytes(del));
// 7 assignment value to the conceal person's deletion object
delDelte.addColumns(Bytes.toBytes(Constants.RELATION_TABLE_CF2), Bytes.toBytes(uid));
// 8 add the conceal person's deletion object into the map
relatDelete.add(delDelte);
}
// 9 add the operator's deletion object into the map
relatDelete.add(uidDelete);
// 10 delete the data in user relationship table
relatTable.delete(relatDelete);
// second operate inbox table
// 1 get the inbox table object
Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));
// 2 create the deletion object of operator
Delete inboxDelete = new Delete(Bytes.toBytes(uid));
// 3 assignment value to operator's deletion object
for (String del : dels) {
inboxDelete.addColumns(Bytes.toBytes(Constants.INBOX_TABLE_CF), Bytes.toBytes(del));
}
// 4 delete data in inbox table
inboxTable.delete(inboxDelete);
// close resource
relatTable.close();
inboxTable.close();
connection.close();
}
4.4 获得用户初始页
// 4 get the initial page data
public static void getInit(String uid) throws IOException {
// 1 get the connection object
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get inbox table object
Table inboxTable = connection.getTable(TableName.valueOf(Constants.INBOX_TABLE));
// 3 get the content table object
Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));
// 4 create inbox get object, and get the data max version
Get inboxGet = new Get(Bytes.toBytes(uid));
inboxGet.setMaxVersions();
Result result = inboxTable.get(inboxGet);
// 5 traverse the data
for (Cell cell : result.rawCells()) {
// 6 construct the content table get object
Get conGet = new Get(CellUtil.cloneValue(cell));
// 7 get the object data
Result conResult = conTable.get(conGet);
// 8 print data
for (Cell conCell : conResult.rawCells()) {
System.out.println("RK:" + Bytes.toString(CellUtil.cloneRow(conCell)) +
", CF:" + Bytes.toString(CellUtil.cloneFamily(conCell)) +
", CN:" + Bytes.toString(CellUtil.cloneQualifier(conCell)) +
", Value:" + Bytes.toString(CellUtil.cloneValue(conCell)));
}
}
// 9 close resource
inboxTable.close();
conTable.close();
connection.close();
}
4.5 获得用户全部微博内容
// 5 get the all the content of somebody weibo
public static void getWeibo(String uid) throws IOException {
// 1 get the connection
Connection connection = ConnectionFactory.createConnection(Constants.CONFIGURATION);
// 2 get the weibo content table
Table conTable = connection.getTable(TableName.valueOf(Constants.CONTENT_TABLE));
// construct scan object
Scan scan = new Scan();
// construct filter
RowFilter rowFilter = new RowFilter(CompareFilter.CompareOp.EQUAL, new SubstringComparator(uid + "_"));
scan.setFilter(rowFilter);
// 4 get the data
ResultScanner resultScanner = conTable.getScanner(scan);
// 5 printf data
for (Result result : resultScanner) {
for (Cell cell : result.rawCells()) {
System.out.println("RK:" + Bytes.toString(CellUtil.cloneRow(cell)) +
", CF:" + Bytes.toString(CellUtil.cloneFamily(cell)) +
", CN:" + Bytes.toString(CellUtil.cloneQualifier(cell)) +
", Value:" + Bytes.toString(CellUtil.cloneValue(cell)));
}
}
// 6 close resource
conTable.close();
connection.close();
}
5 test包 测试
在test包当中，编写测试代码，相对简单。
package test;
import constants.Constants;
import dao.HBaseDao;
import utils.HBaseUtil;
import java.io.IOException;
public class TestWeiBo {
public static void init() {
try {
// create namespace
HBaseUtil.createNameSpace(Constants.NAMESPACE);
// create weibo content table
HBaseUtil.createTable(Constants.CONTENT_TABLE, Constants.CONTENT_TABLE_VERSIONS, Constants.CONTENT_TABLE_CF);
// create user relationship table
HBaseUtil.createTable(Constants.RELATION_TABLE, Constants.RELATION_TABLE_VERSIONS,
Constants.RELATION_TABLE_CF1, Constants.RELATION_TABLE_CF2);
// create inbox table
HBaseUtil.createTable(Constants.INBOX_TABLE, Constants.INBOX_TABLE_VERSIONS, Constants.INBOX_TABLE_CF);
} catch (IOException e) {
e.printStackTrace();
}
}
public static void main(String[] args) throws IOException, InterruptedException {
// initialize
init();
System.out.println("init done");
// 1001 publish weibo
HBaseDao.publishWeiBo("1001","这是1001的微博，Hello World");
System.out.println("publish done");
// 1002 attend 1001, 1003
HBaseDao.addAttends("1002", "1001", "1003");
System.out.println("attend done");
// get the 1002 initial page
HBaseDao.getInit("1002");
System.out.println("*********************111**********************");
// 1003 publish 3 weibo, at the same time 1002 publish 2 weibo
HBaseDao.publishWeiBo("1003", "快结束了HBase了！！");
Thread.sleep(10);
HBaseDao.publishWeiBo("1001", "那你还是个弟弟啊！！");
Thread.sleep(10);
HBaseDao.publishWeiBo("1003", "哦，不我要当哥哥！！");
Thread.sleep(10);
HBaseDao.publishWeiBo("1001", "弟弟！！");
Thread.sleep(10);
HBaseDao.publishWeiBo("1003", "叫哥哥！！");
// get 1002 initial page
HBaseDao.getInit("1002");
System.out.println("*********************222***********************");
// 1002 conceal 1003
HBaseDao.deleteAttends("1002", "1003");
// get the 1002 initial page
HBaseDao.getInit("1002");
System.out.println("*********************333***********************");
// 1002 attend 1003 again
HBaseDao.addAttends("1002", "1003");
// get the 1002 initial page
HBaseDao.getInit("1002");
System.out.println("*********************444***********************");
// get 1001 detail initial page
HBaseDao.getWeibo("1001");
System.out.println("*********************555***********************");
}
}
总结
不知道为什么在configuration配置的时候总是存在问题，最后参考以前写的测试API的代码，加上了configuration.set()，才可以跑通。否则一直显示的是连接超时，不存在zookeeper的master。具体问题还不清楚，感觉像是代码找不到连接zookeeper的端口了。
最后我这个弱鸡终于成功跑通了代码，感觉还是蛮不错的，记录一下遇到的坑。
1、编写代码的时候，需要记住addColumns()函数里面是需要对列进行操作，好几次写成了table，忘了加CF了，然后报错。并且还需要转换成字节的形式。
2、需要弄清楚业务的逻辑，因为里面只是设计了三张表所以逻辑相对简单。但是尽可能的还是多考虑考虑一些解耦合的操作或者是需求分析的时候对创建表考虑清楚。
最后，加油吧，诸君。
501
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
