java 队列 写日志_logback KafkaAppender 写入Kafka队列,集中日志输出.

为了减少应用服务器对磁盘的读写,以及可以集中日志在一台机器上,方便使用ELK收集日志信息,所以考虑做一个jar包,让应用集中输出日志

网上搜了一圈,只发现有人写了个程序在github

本来打算引用一下这个jar就完事了,没想到在pom里下不下来,只好把源码下了,拷贝了代码过来,自己修改一下.

首先,安装一个Kafka,作为一个懒得出神入化得程序员,我选择的安装方式是

启动zookeeper容器

docker run -d --name zookeeper --net=host -t wurstmeister/zookeeper

启动kafka容器

docker run --name kafka -d -e HOST_IP=192.168.1.7 --net=host -v /usr/local/docker/kafka/conf/server.properties:/opt/kafka_2.12-1.0.0/config/server.properties -v /etc/localtime:/etc/localtime:ro -e KAFKA_ADVERTISED_PORT=9092 -e KAFKA_BROKER_ID=1 -t wurstmeister/kafka

要修改Kafka的server.properties 中zookeeper配置

配置文件如下

listeners=PLAINTEXT://192.168.1.7:9092

delete.topic.enable=trueadvertised.listeners=PLAINTEXT://192.168.1.7:9092num.network.threads=3num.io.threads=8socket.send.buffer.bytes=102400# The receive buffer (SO_RCVBUF) used by the socket server

socket.receive.buffer.bytes=102400# The maximum size of a request that the socket server will accept (protection against OOM)

socket.request.max.bytes=104857600log.dirs=/kafka/kafka-logs-92cfb0bbd88c

num.partitions=1num.recovery.threads.per.data.dir=1offsets.topic.replication.factor=1transaction.state.log.replication.factor=1transaction.state.log.min.isr=1log.retention.hours=168log.retention.bytes=10737418240log.segment.bytes=1073741824log.retention.check.interval.ms=300000zookeeper.connect=192.168.1.7:2181# Timeoutin ms forconnecting to zookeeper

zookeeper.connection.timeout.ms=6000group.initial.rebalance.delay.ms=0version=1.0.0

启动好了,新建SpringBoot项目,首先消费队列的

pom文件

4.0.0

com.lzw

kafkalog

0.0.1-SNAPSHOT

jar

kafkalog

Demo project for Spring Boot

org.springframework.boot

spring-boot-starter-parent

2.0.0.M6

UTF-8

UTF-8

1.8

org.springframework.kafka

spring-kafka

org.springframework.boot

spring-boot-starter

org.springframework.boot

spring-boot-starter-test

test

org.springframework.boot

spring-boot-maven-plugin

spring-snapshots

Spring Snapshots

https://repo.spring.io/snapshot

true

spring-milestones

Spring Milestones

https://repo.spring.io/milestone

false

spring-snapshots

Spring Snapshots

https://repo.spring.io/snapshot

true

spring-milestones

Spring Milestones

https://repo.spring.io/milestone