简介
通过mybatis统计出当天日活数据,返回Json数据
一 项目实时架构图
二 准备
2.1 启动服务端的应用程序如下
- zookeeper :kafka hbase 依赖zookeeper
- hadoop :hbase 数据存储在hdfs上
- nginx :反向代理gmall-logger springboot程序
- redis :实时处理模块,数据去重
- Phoenix :通过phoenix将去重后的数据写入hbase
- hbase:存储日活数据
2.2 启动本地程序如下:
- gmall-realtime (实时处理)项目:消费kafka中主题为启动日志的数据
- gmall-moke(模拟数据):生成数据,通过post方式发送到Nginx
- gamll-publish(接口模块):整合mybatis与Phoenix,通过Phoenix查询Hbase中的数据,返回Json数据
2.3 pom文件
<parent>
<artifactId>gmall</artifactId>
<groupId>com.gc.spark</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<groupId>com.gc</groupId>
<artifactId>gmall-publish</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>gmall-publish</name>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.mybatis.spring.boot</groupId>
<artifactId>mybatis-spring-boot-starter</artifactId>
<version>2.1.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.phoenix</groupId>
<artifactId>phoenix-core</artifactId>
<version>4.14.2-HBase-1.3</version>
</dependency>
<dependency>
<groupId>com.gc.spark</groupId>
<artifactId>gmall-common</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
2.4 application.properties配置
server.port=8070
logging.level.root=error
# jdbc
spring.datasource.driver-class-name=org.apache.phoenix.jdbc.PhoenixDriver
spring.datasource.url=jdbc:phoenix:hadoop102,hadoop103,hadoop104:2181
spring.datasource.data-username=
spring.datasource.data-password=
# mybatis
# 做映射的时候会在 resources/mapper 目录找对应的 xml 文件
mybatis.mapperLocations=classpath:mapper/*.xml
# 字段名是否自动从下划线映射到驼峰命名 一般都是使用 true
mybatis.configuration.map-underscore-to-camel-case=true
三 项目分层配置
3.1 总体项目介绍
3.2 gmall-publish 项目
3.3 mapper
package com.gc.gmallpublish.mapper;
import java.util.List;
import java.util.Map;
public interface DauMapper {
// 根据日期查询指定的日活
long queryDauByDate(String date);
// 查询指定日期下,每个小时分段统计的结果
List<Map> queryDauByHour(String date);
}
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper SYSTEM "http://mybatis.org/dtd/mybatis-3-mapper.dtd" >
<mapper namespace="com.gc.gmallpublish.mapper.DauMapper">
<select id="queryDauByDate" resultType="java.lang.Long">
select count(1) as count from GMALL_DAU where LOGDATE=#{date}
</select>
<select id="queryDauByHour" resultType="java.util.Map">
select LOGHOUR,count(1) as COUNT from GMALL_DAU where LOGDATE=#{date} group by LOGHOUR
</select>
</mapper>
三 测试接口
四 总结
- Phoenix连接注意连接地址的写法
- 表和字段的大小写
- 启动类加扫描包