参考文章:https://blog.csdn.net/qiaqia609/article/details/78899298
服务器的kafka是配置在docker内的,由于日志越来越多,需要清理空间,
目标是把kafka的日志文件写到nas的服务器上去。
(一)进入kafka容器,删除log文件
1.docker ps 获取到当前kafka的容器id:【5fdfc4397314】
docker ps -a 可以查看所有容器,docker stats,查看吞吐量
du -sh,统计当前目录占用空间,df -h 查看磁盘空间。
[root@izbp1d36xiav554wzwc3klz]/mnt/mntnas/estestproject/bin# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5fdfc4397314 wurstmeister/kafka "start-kafka.sh" 3 hours ago Up 22 minutes 0.0.0.0:32769->9092/tcp kafka_kafka_1
9fdbf61ddef8 sheepkiller/kafka-manager "./start-kafka-manag…" 4 weeks ago Up 22 minutes 0.0.0.0:39000->9000/tcp kafka_kafka-manager_1
656872ae9ea5 docker-elk_kibana "/usr/local/bin/kiba…" 4 weeks ago Up 3 weeks 0.0.0.0:5601->5601/tcp docker-elk_kibana_1
2645957e420f docker-elk_logstash "/usr/local/bin/dock…" 4 weeks ago Up 3 weeks 0.0.0.0:5000->5000/tcp, 0.0.0.0:9600->9600/tcp, 5044/tcp docker-elk_logstash_1
df5cf47b0ae0 docker-elk_elasticsearch "/usr/local/bin/dock…" 4 weeks ago Up 3 weeks 0.0.0.0:9200->9200/tcp, 0.0.0.0:9300->9300/tcp docker-elk_elasticsearch_1
0f710b861004 wurstmeister/zookeeper "/bin/sh -c '/usr/sb…" 6 weeks ago Up 5 hours 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp kafka_zookeeper_1
2.进入容器,docker exec -it 5fdfc4397314 bash ,找到topic的文件夹【erplog-012345】
[root@izbp1d36xiav554wzwc3klz]/mnt/mntnas/estestproject/bin# docker exec -it 5fdfc4397314 bash
bash-4.4# cd /kafka/KafkaLog
bash-4.4# ls -la
total 24
drwxrwxr-x 8 42949672 42949672 4096 Aug 16 14:48 .
drwxr-xr-x 3 root root 4096 Aug 16 11:26 ..
-rw-r--r-- 1 42949672 42949672 0 Aug 16 11:24 .lock
-rw-r--r-- 1 42949672 42949672 0 Aug 16 11:24 cleaner-offset-checkpoint
drwxr-xr-x 2 42949672 42949672 4096 Aug 16 14:22 erplog-0
drwxr-xr-x 2 42949672 42949672 4096 Aug 16 14:23 erplog-1
drwxr-xr-x 2 42949672 42949672 4096 Aug 16 14:23 erplog-2
drwxr-xr-x 2 42949672 42949672 4096 Aug 16 14:23 erplog-3
drwxr-xr-x 2 42949672 42949672 4096 Aug 16 14:23 erplog-4
drwxr-xr-x 2 42949672 42949672 4096 Aug 16 14:23 erplog-5
-rw-r--r-- 1 42949672 42949672 4 Aug 16 14:48 log-start-offset-checkpoint
-rw-r--r-- 1 42949672 42949672 57 Aug 16 11:24 meta.properties
-rw-r--r-- 1 42949672 42949672 70 Aug 16 14:48 recovery-point-offset-checkpoint
-rw-r--r-- 1 42949672 42949672 89 Aug 16 14:48 replication-offset-checkpoint
bash-4.4# rm -rf erpxxxxx
3.rm -rf erplog-012345,删除这些目录
(二)去kafka-manager的管理页面,删掉erplog这个topic,也可以命令删除。
(三)去zookeeper里删除topic=erplog
1.先停止kafka,kafkamanager程序
docker stop kafka-id
docker stop kafkamanager-id
2.进入zookeeper容器:docker exec -it zookeeper-id bash
3.进入zookeeper服务:./zkCli.sh -server 127.0.0.1:2181
4.列出topic:ls /brokers/topics
5.删除topic:rmr /brokers/topics/erplog
root@0f710b861004:/opt/zookeeper-3.4.13# cd bin
root@0f710b861004:/opt/zookeeper-3.4.13/bin# ./zkCli.sh -server 127.0.0.1:2181
Connecting to 127.0.0.1:2181
2019-07-10 08:17:17,474 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT
2019-07-10 08:17:17,476 [myid:] - INFO [main:Environment@100] - Client environment:host.name=0f710b861004
2019-07-10 08:17:17,476 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.7.0_65
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/opt/zookeeper-3.4.13/bin/../build/classes:/opt/zookeeper-3.4.13/bin/../build/lib/*.jar:/opt/zookeeper-3.4.13/bin/../lib/slf4j-log4j12-1.7.25.jar:/opt/zookeeper-3.4.13/bin/../lib/slf4j-api-1.7.25.jar:/opt/zookeeper-3.4.13/bin/../lib/netty-3.10.6.Final.jar:/opt/zookeeper-3.4.13/bin/../lib/log4j-1.2.17.jar:/opt/zookeeper-3.4.13/bin/../lib/jline-0.9.94.jar:/opt/zookeeper-3.4.13/bin/../lib/audience-annotations-0.5.0.jar:/opt/zookeeper-3.4.13/bin/../zookeeper-3.4.13.jar:/opt/zookeeper-3.4.13/bin/../src/java/lib/*.jar:/opt/zookeeper-3.4.13/bin/../conf:
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib/x86_64-linux-gnu/jni:/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/usr/lib/jni:/lib:/usr/lib
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2019-07-10 08:17:17,479 [myid:] - INFO [main:Environment@100] - Client environment:os.version=4.4.95-2.al7.x86_64
2019-07-10 08:17:17,480 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2019-07-10 08:17:17,480 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2019-07-10 08:17:17,480 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/opt/zookeeper-3.4.13/bin
2019-07-10 08:17:17,481 [myid:] - INFO [main:ZooKeeper@442] - Initiating client connection, connectString=127.0.0.1:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@674e5e21
Welcome to ZooKeeper!
2019-07-10 08:17:17,503 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1029] - Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2019-07-10 08:17:17,521 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@879] - Socket connection established to localhost/127.0.0.1:2181, initiating session
2019-07-10 08:17:17,531 [myid:] - INFO [main-SendThread(localhost:2181):ClientCnxn$SendThread@1303] - Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x1008c3b12000004, negotiated timeout = 30000
WATCHER::
WatchedEvent state:SyncConnected type:None path:null
[zk: 127.0.0.1:2181(CONNECTED) 0] ls /brokers/topics
[__consumer_offsets, erplog]
[zk: 127.0.0.1:2181(CONNECTED) 4] rmr /brokers/topics/erplog
[zk: 127.0.0.1:2181(CONNECTED) 6] quit
Quitting...
2019-07-10 08:23:16,881 [myid:] - INFO [main:ZooKeeper@693] - Session: 0x1008c3b12000004 closed
2019-07-10 08:23:16,882 [myid:] - INFO [main-EventThread:ClientCnxn$EventThread@522] - EventThread shut down for session: 0x1008c3b12000004
root@0f710b861004:/opt/zookeeper-3.4.13/bin#
6.删除好了,依次重启服务
docker restart zookeeper-id
docker start kafka-id
docker start kafka-manager-id