文章目录
HBase实战之谷粒微博
需求分析
-
微博内容的浏览,数据库表设计
-
用户社交体现:关注用户,取关用户
-
拉取关注的人的微博内容
表的分类
1.微博内容表
表结构:
方法名 | creatTableeContent |
---|---|
Table Name | weibo:content |
RowKey | 用户 ID_时间戳 |
ColumnFamily | info |
ColumnLabel | 标题,内容,图片 |
Version | 1 个版本 |
2.用户关系表
表结构:
方法名 | createTableRelations |
---|---|
Table Name | weibo:relation |
RowKey | 用户 ID |
ColumnFamily | attends、fans |
ColumnLabel | 关注用户 ID,粉丝用户 ID |
ColumnValue | 用户 ID |
Version | 1 个版本 |
3.微博收件箱表
表结构:
方法名 | createTableReceiveContentEmails |
---|---|
Table Name | weibo:inbox |
RowKey | 用户 ID |
ColumnFamily | info |
ColumnLabel | 用户 ID |
ColumnValue | 取微博内容的 RowKey |
Version | 2 |
代码实现
1.代码设计总览:
-
创建命名空间以及表名的定义
-
创建微博内容表
-
创建用户关系表
-
创建用户微博内容接收邮件表
-
发布微博内容
-
添加关注用户
-
移除(取关)用户
-
获取关注的人的微博内容
-
测试
2. 环境搭建
1.pom文件
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>Hadoop</artifactId>
<groupId>com.huan</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>guli-weibo</artifactId>
<dependencies>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-server</artifactId>
<version>1.3.1</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.3.1</version>
</dependency>
</dependencies>
</project>
2.包名规则
3.resources文件 (hbase-site.xml 可以直接在本地运行,无需打包jar)
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
-->
<configuration>
<property>
<name>hbase.rootdir</name>
<value>hdfs://Bigdata01:9000/hbase</value>
</property>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<!-- 0.98后的新变动,之前版本没有.port,默认端口为60000 -->
<property>
<name>hbase.master.port</name>
<value>16000</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>Bigdata01:2181,Bigdata02:2181,Bigdata03:2181</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/opt/module/zookeeper-3.4.10/zkData</value>
</property>
<property>
<name>hbase.master.maxclockskew</name>
<value>180000</value>
<description>Time difference of regionserver from master</description>
</property>
</configuration>
3.创建命名空间以及表名的定义
这里我们设置一个常量给它们定义好方便后续的使用,在后续使用时直接调用即可:
package com.huan.constants;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
public class Constants {
//TODO 定义上下文配置信息
public static final Configuration CONFIGURATION = HBaseConfiguration.create();
//TODO 定义命名空间
public static final String NAMESPACE = "weibo";
//TODO 微博内容表
public static final String CONTENT_TABLE = "weibo:content";
public static final String CONTENT_TABLE_CF = "info";
public static final int CONTENT_TABLE_VERSION = 1;
//TODO 用户关系表
public static final String RELATION_TABLE = "weibo:relation";
public static final String RELATION_TABLE_CF1 = "attends";
public static final String RELATION_TABLE_CF2 = "fans";