项目效果概览:
项目架构:
获取文件模板,前端就会展示问卷的列表,我们就可以选择选项,结果就会同步到我们的后端,通过后端把统计结果发生到前端,完成整个流程.
git仓库地址:
目录标题
1.基础准备工作
1.1lombok插件
lomlok的作用主要是简化代码开发
1.2maven依赖配置
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.5.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.imooc.jiangzh</groupId>
<artifactId>kafka-study</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>kafka-study</name>
<description>微信调查问卷</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<!--springBoot依赖两个--->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>cn.hutool</groupId>
<artifactId>hutool-all</artifactId>
<version>4.1.1</version>
</dependency>
<!--kafka相关依赖两个-->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-clients</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-streams</artifactId>
<version>2.4.0</version>
</dependency>
<!-- lombok依赖 -->
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.12</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>28.2-jre</version>
</dependency>
<!-- fastjson依赖 -->
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.68</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
1.3 接口设计
1.4模块式开发
成品样式:
1.5common公共类
1.5.1BaseResponseVO:实现的是公共返回值
用泛型去做枚举,当需要真实返回值的时候按照枚举类型进行传入就可以,当不需返回值的时候,只有RequestId
package com.yuge.wechat.questionnaire.common;
import java.util.UUID;
import lombok.Data;
/**
* @description : 公共返回对象
**/
@Data
public class BaseResponseVO<M> {
private String requestId;
private M result;
public static<M> BaseResponseVO success(){
BaseResponseVO baseResponseVO = new BaseResponseVO();
baseResponseVO.setRequestId(getRequestId());
return baseResponseVO;
}
public static<M> BaseResponseVO success(M result){
BaseResponseVO baseResponseVO = new BaseResponseVO();
baseResponseVO.setRequestId(getRequestId());
baseResponseVO.setResult(result);
return baseResponseVO;
}
private static String getRequestId(){
return UUID.randomUUID().toString();
}
}
15.2CorsFilter:解决跨域问题
package com.yuge.wechat.questionnaire.common;
import java.io.IOException;
import javax.servlet.Filter;
import javax.servlet.FilterChain;
import javax.servlet.ServletException;
import javax.servlet.ServletRequest;
import javax.servlet.ServletResponse;
import javax.servlet.annotation.WebFilter;
import javax.servlet.http.HttpServletResponse;
import org.springframework.context.annotation.Configuration;
/**
* @description : 跨域问题解决
**/
@WebFilter(filterName = "CorsFilter")
@Configuration
public class CorsFilter implements Filter {
@Override
public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
HttpServletResponse response = (HttpServletResponse) res;
response.setHeader("Access-Control-Allow-Origin","*");
response.setHeader("Access-Control-Allow-Credentials", "true");
response.setHeader("Access-Control-Allow-Methods", "POST, GET, PATCH, DELETE, PUT");
response.setHeader("Access-Control-Max-Age", "3600");
response.setHeader("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
chain.doFilter(req, res);
}
}
1.6utils工具类
package com.yuge.wechat.questionnaire.utils;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.io.Reader;
import java.util.Optional;
import lombok.Cleanup;
import lombok.extern.slf4j.Slf4j;
/**
* @description : 文件工具类
* 给一个路径filePath,返回路径下内容,重载了多种返回类型:string,JSONObject,JSONArray
**/
@Slf4j
public class FileUtils {
public static String readFile(String filePath) throws IOException {
@Cleanup
BufferedReader reader = new BufferedReader(
new FileReader(new File(filePath))
);
String lineStr = "";
StringBuffer stringBuffer = new StringBuffer();
while ((lineStr = reader.readLine()) != null) {
stringBuffer.append(lineStr);
}
return stringBuffer.toString();
}
public static Optional<JSONObject> readFile2JsonObject(String filePath){
try {
String fileContent = readFile(filePath);
log.info("readFile2Json fileContent: [{}]" , fileContent);
return Optional.ofNullable(JSON.parseObject(fileContent));
} catch (IOException e) {
e.printStackTrace();
}
return Optional.empty();
}
public static Optional<JSONArray> readFile2JsonArray(String filePath){
try {
String fileContent = readFile(filePath);
log.info("readFile2Json fileContent: [{}]" , fileContent);
return Optional.ofNullable(JSON.parseArray(fileContent));
} catch (IOException e) {
e.printStackTrace();
}
return Optional.empty();
}
}
1.7conf配置读取类
package com.yuge.wechat.questionnaire.conf;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
import java.util.List;
@Data
@Configuration
@ConfigurationProperties(prefix = "template")
public class WechatTemplateProperties {
private List<WetchTemplate> templates;
private int templateResultType; //0-文件夹 1-数据库
private String templateResultFilePath;//结果文件路径
@Data
public static class WetchTemplate{
private String templateId; //模板编号,用来模板对应接口的唯一标识
private String templateFilePath;
private String active;
}
}
1.7.1配置application.yml文件
- 注意:conf类下定义的变量是驼峰命名templateResultType到application需要转成tempalte_result_type
server:
port: 8080
template:
templates:
- {"templateId":"1","templateFilePath":"C:/user/MyCode/wechat-questionnaire/src/main/resources/template/template.json","active":true}
- {"templateId":"2","templateFilePath":"C:/user/MyCode/wechat-questionnaire/src/main/resources/template/template.json","active":false}
template_result_type: 0
template_result_filePath: "C:/user/MyCode/wechat-questionnaire/src/main/resources/template/templateRestlt.json"
注意事项:路径是用反斜杠的,而反斜杠在java中是转义符所有把把反斜杠换成/才可以
这一个json是由多个数组组成的,一个数组就一个文件的题目和选项
前端展示的时候他会通过接口调这个文件的内容,文件是什么内容他就会展示什么内容
1.7.2做一个测试的模板
[{ "questionId": "1", //问题编号
"question": "今天几号", //问题内容
"answer": "", //默认答案
"options": [ //选项
{"label": "1 号", "value": "A"},
{"label": "2 号", "value": "B"},
{"label": "3 号", "value": "C"},
{"label": "4 号", "value": "D"}
]},
{"questionId": "2",
"question": "你喜爱的颜色",
"answer": "",
"options": [
{"label": "红色", "value": "A"},
{"label": "黄色", "value": "B"},
{"label": "绿色", "value": "C"},
{"label": "紫色", "value": "D"}
]}
]
{ "templateId": "001", //模板编号
"totalNumber": "102", //有多少人作答
"statistics": [ { //针对这个templateId我们统计的结果是什么
"questionId": "1",
"question": "今天几号",
"answers": [
{"label": "A", "value": 10}, //选a的10人 ,选b的50人,选c的12人,选d的17人
{"label": "B", "value": 50},
{"label": "C", "value": 12},
{"label": "D", "value": 17}
]
}, {
"questionId": "2",
"question": "你喜爱的颜色",
"answers": [
{"label": "A", "value": 12},
{"label": "B", "value": 52},
{"label": "C", "value": 17},
{"label": "D", "value": 17} ]
}
]
}
一个存储模板一个存储模板的结果
2.微信调查问卷模板配置
3.业务层实现(业务层实现在service包下)
3.1接口WechatTemplateService,主要作用是提供controller进行调取的,真正实现方法在接口的实现类中
package com.yuge.wechat.questionnaire.service;
import com.alibaba.fastjson.JSONObject;
import com.yuge.wechat.questionnaire.conf.WechatTemplateProperties;
public interface WechatTemplateservice {
//前端获取统计的模板 获取active为true的模板
WechatTemplateProperties.WetchTemplate getWeChatQuestionnaireTemplate();
//前端返回调查问卷结果数据到后端
void reportTheResultsOfTheQuestionnaire(JSONObject resultsData);
//后端把统计结果数据发送给前端
JSONObject questionnaireStatistics(String templateId);
}
3.2接口WechatTemplateService的实现类,通过此类实现接口中定义的方法
package com.yuge.wechat.questionnaire.service;
import com.alibaba.fastjson.JSONObject;
import com.yuge.wechat.questionnaire.conf.WechatTemplateProperties;
import com.yuge.wechat.questionnaire.utils.FileUtils;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import java.util.List;
import java.util.Optional;
@Slf4j //lombok提供的日志
@Service
public class WechatTemplateServiceImpl implements WechatTemplateservice {
@Autowired
WechatTemplateProperties properties;
/**
* 前端向后端获取微信调查问卷为active模板
* @return
*/
@Override
public WechatTemplateProperties.WetchTemplate getWeChatQuestionnaireTemplate() {
List<WechatTemplateProperties.WetchTemplate> templates = properties.getTemplates();
Optional<WechatTemplateProperties.WetchTemplate> wetchTemplate = templates.stream().filter((template) -> template.isActive()).findFirst();
return wetchTemplate.isPresent()? wetchTemplate.get():null;
}
/**
* 前端问卷结果返回后端
* @param resultsData
*/
@Override
public void reportTheResultsOfTheQuestionnaire(JSONObject resultsData) {
//kafka Producer 将数据推倒topic
log.info("reportTheResultsOfTheQuestionnaire:[{}]",resultsData);
}
/**
* 后端前前端发送调查问卷统计结果
* @param templateId
* @return
*/
@Override
public JSONObject questionnaireStatistics(String templateId) {
if (templateId=="0"){ //0就文件获取
return FileUtils.readFile2JsonObject(properties.getTemplateResultFilePath()).get();
}else{
//其他数据源获取
return null;
}
}
}
4.表现层controller实现
实现的是三个接口逻辑
package com.yuge.wechat.questionnaire.controller;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.JSONArray;
import com.alibaba.fastjson.JSONObject;
import com.yuge.wechat.questionnaire.common.BaseResponseVO;
import com.yuge.wechat.questionnaire.conf.WechatTemplateProperties;
import com.yuge.wechat.questionnaire.service.WechatTemplateservice;
import com.yuge.wechat.questionnaire.utils.FileUtils;
import org.apache.kafka.common.protocol.types.Field;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.*;
import java.util.HashMap;
@Controller
@RequestMapping("/v1")
public class WechatTemplateController {
@Autowired
WechatTemplateservice wechatTemplateservice;
@Autowired
WechatTemplateProperties properties;
/**
* 获取调查问卷模板的数据(前端获取一个问卷内容,供客户来填写)
*/
@RequestMapping(value = "/getTheQuestionnaireTemplate",method = RequestMethod.GET)
public BaseResponseVO getTheQuestionnaireTemplate(){
WechatTemplateProperties.WetchTemplate wetchTemplate = wechatTemplateservice.getWeChatQuestionnaireTemplate();
HashMap<String, Object> map = new HashMap<>();
map.put("templateId",wetchTemplate.getTemplateId());
map.put("template", FileUtils.readFile2JsonArray(properties.getTemplateResultFilePath()));
return BaseResponseVO.success(map);
}
/**
* 返回到后端的问卷填写数据,
*/
@RequestMapping(value = "/returnQuestionnaireResults",method = RequestMethod.POST)
public BaseResponseVO returnQuestionnaireResults(@RequestBody String resultData){
wechatTemplateservice.reportTheResultsOfTheQuestionnaire(JSON.parseObject(resultData));
return BaseResponseVO.success();
}
/**
* 返回数据的统计结果,允许前端传入模板编号
*/
@RequestMapping(value = "/getQuestionnaireStatistics",method = RequestMethod.GET)
public BaseResponseVO getQuestionnaireStatistics(@RequestParam(value="templateId",required = false) String templateId){
JSONObject statistics = wechatTemplateservice.questionnaireStatistics(templateId);
return BaseResponseVO.success(statistics);
}
}
5.程序业务测试
-- 查询模板信息
curl -XGET http://localhost:8080/v1/template
-- 查询模板统计结果
curl -XGET http://localhost:8080/v1/template/result
-- 传入调查问卷结果
curl -XPOST -H "Content-Type:application/json; charset=UTF-8" http://localhost:8080/v1/template/report -d \
'{
templateId:"001",
result:[
{"questionId":"1","question":"今天几号","answer":"A"},
{"questionId":"2","question":"你喜爱的颜色","answer":"B"}
]
}'
netstat -ano | findstr 443
6.Kafka Producer集成
package com.imooc.jiangzh.kafka.wechat.conf;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.annotation.Configuration;
@Data
@Configuration
@ConfigurationProperties(prefix = "wechat.kafka")
public class KafkaProperties {
private String bootstrapServers;
private String acksConfig;
private String retriesConfig;
private String batchSizeConfig;
private String lingerMsConfig;
private String bufferMemoryConfig;
private String keySerializerClassConfig;
private String valueSerializerClassConfig;
}
package com.imooc.jiangzh.kafka.wechat.conf;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.Producer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.Properties;
@Configuration
public class KafkaConf {
@Autowired
private KafkaProperties kafkaProperties;
@Bean//增加@Bean注释:依赖spring提供的上下文,来做一个单例模式,这样我们所以的线程都是共享同一个kafkaProducer
public Producer kafkaProducer(){
Properties properties = new Properties();
properties.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, kafkaProperties.getBootstrapServers());
properties.put(ProducerConfig.ACKS_CONFIG, kafkaProperties.getAcksConfig());
properties.put(ProducerConfig.RETRIES_CONFIG,kafkaProperties.getRetriesConfig());
properties.put(ProducerConfig.BATCH_SIZE_CONFIG,kafkaProperties.getBatchSizeConfig());
properties.put(ProducerConfig.LINGER_MS_CONFIG,kafkaProperties.getLingerMsConfig());
properties.put(ProducerConfig.BUFFER_MEMORY_CONFIG,kafkaProperties.getBufferMemoryConfig());
properties.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG,kafkaProperties.getKeySerializerClassConfig());
properties.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG,kafkaProperties.getValueSerializerClassConfig());
// Producer的主对象
Producer<String,String> producer = new KafkaProducer<>(properties);
return producer;
}
}
7.HTTPS的支持(因为微信小程序要求必须是HTTPS)
7.1CA证书申请
7.2域名绑定
8.集成SSL证书
9.阿里云部署后端
10.编译部署
11.完整项目代码git仓库地址
12.项目后续扩展
上述只是简单的实现了微信小程序的调查问卷功能:
后期项目迭代增加需求:
1.通过mybaits依赖增加mysql数据库支持
2.增加大数据分析大屏数据展示
12.1BigData实现
我们通过kafkaProducer把数据发送到topic上面,但是数据并未做任何分析和展示,后期我们需要做一个数据大屏,来统计分析文件结果,并直观展示.
- 架构:kafkaProducer->consumer->hbase->sparkSQL->hbase->Kylin(数据展示)