1. 实现目的及原理介绍
(1)html作为用户操作页面,通过操作发送请求到nginx
(2)将ngx_kafka_module模块编译进nginx,使得nginx的消息能直接发送到kafka
(3)搭建kafka,进行接收日志消息
2.软件介绍
环境&软件 | 版本 |
---|---|
虚拟机&VMware Workstation Pro | VMwareworkstation15.5.6 |
服务器&Centos | CentOS-7-x86_64-DVD-1810.iso |
jdk | jdk-8u261-linux-x64.rpm |
zookeeper | zookeeper-3.4.14.tar.gz |
kafka | kafka_2.12-1.0.2.tgz |
nginx | nginx-1.18.0.tar.gz |
3.安装步骤
(1) 解压jdk,配置环境变量
rpm -ivh jdk-8u261-linux-x64.rpm
#配置环境变量
vi /etc/profile
export JAVA_HOME=/usr/java/jdk1.8.0_261-amd64
export PATH=$PATH:$JAVA_HOME/bin
(2)安装zookeeper
#解压到opt目录
tar -zxf zookeeper-3.4.14.tar.gz -C /opt
cd /opt/zookeeper-3.4.14/conf
# 复制zoo_sample.cfg命名为zoo.cfg
cp zoo_sample.cfg zoo.cfg
# 编辑zoo.cfg文件
vi zoo.cfg
dataDir=/var/zookeeper/data
#配置环境变量
export ZOOKEEPER_PREFIX=/opt/zookeeper-3.4.14
export PATH=$PATH:$ZOOKEEPER_PREFIX/bin
export ZOO_LOG_DIR=/var/zookeeper/log
(3) 安装nginx
#解压nginx
tar -zxvf nginx-1.18.0.tar.gz -C /opt
#使用git命令下载librdkafka,这个是ngx_kafka_module所需依赖,我这边是把这些下载到/opt目录下
cd /opt
git clone https://github.com/edenhill/librdkafka
cd librdkafka
#执行下面命令可能需要gcc,需自行安装下
./configure
make
make install
#下载ngx_kafka_module模块到/opt,编译到解压的nginx中
cd /opt
git clone https://github.com/brg-liuwei/ngx_kafka_module
# 进入到nginx目录,进行编译并安装nginx
cd /opt/nginx-1.18.0
./configure --add-module=/opt/ngx_kafka_module
make #编译
make install #安装
配置nginx
#安装后的nginx在/usr/local/nginx里
#在nginx配置文件里增加kafka配置,在对应节点内增加
http {
kafka;
#指定kafka地址
kafka_broker_list 127.0.0.1:9092 127.0.0.1:9093; # host:port ...
server {
location = /your/path/topic {
# optional directive: kafka_partition [<partition-num> | auto]
#
# kafka_partition auto; # default value
# kafka_partition 0;
# kafka_partition 1;
#指定主题名称
kafka_topic your_topic;
}
}
}
(4)安装kafka
tar -zxf kafka_2.12-1.0.2.tgz -C /opt
#配置环境变量
export KAFKA_HOME=/opt/kafka_2.12-1.0.2
export PATH=$PATH:$KAFKA_HOME/bin
#配置/opt/kafka_2.12-1.0.2/config中的server.properties文件,修改配置
zookeeper.connect=localhost:2181/myKafka
4.启动服务
#重新加载profile文件,使环境变量生效
source /etc/profile
#启动zookeeper
zkServer.sh start
#启动nginx
nginx
#启动kafka
kafka-server-start.sh /opt/kafka_2.12-1.0.2/config/server.properties
静态html页面
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>职位浏览</title>
<script src="https://apps.bdimg.com/libs/jquery/2.1.4/jquery.min.js">
</script>
</head>
<body>
<table>
<tr>
<th>职位</th>
<th>企业</th>
<th>薪资</th>
<th>操作</th>
</tr>
<tr>
<td>开发</td>
<td>阿里</td>
<td>10000</td>
<td>
<button onclick="operation('A001', 'click')">点击</button>
<button onclick="operation('A001', 'job_collect')">收藏</button>
<button onclick="operation('A001', 'cv_send')">投递简历</button>
<button onclick="operation('A001', 'cv_upload')">上传简历</button>
</td>
</tr>
<tr>
<td>开发</td>
<td>腾讯</td>
<td>20000</td>
<td>
<button onclick="operation('A002', 'click')">点击</button>
<button onclick="operation('A002', 'job_collect')">收藏</button>
<button onclick="operation('A002', 'cv_send')">投递简历</button>
<button onclick="operation('A002', 'cv_upload')">上传简历</button>
</td>
</tr>
<tr>
<td>开发</td>
<td>拉勾</td>
<td>30000</td>
<td>
<button onclick="operation('A003', 'click')">点击</button>
<button onclick="operation('A003', 'job_collect')">收藏</button>
<button onclick="operation('A003', 'cv_send')">投递简历</button>
<button onclick="operation('A003', 'cv_upload')">上传简历</button>
</td>
</tr>
</table>
</body>
<script>
function operation(job_code,action) {
$.ajax({
url: 'http://192.168.5.141/your/path/topic',
type: 'POST',
contentType: 'application/json;charset=utf-8',
dataType: 'json',
data:{
user_id:101,
act_time: new Date().getTime(),
action:action,
job_code:job_code
},
success: function (data) {
}
})
}
</script>
</html>
将html文件放到nginx下的html目录里
5.效果演示
kafka消费端监听到的消息