flume日志、zookeeper偏移量查询与设置

1.查询flume日志(INFO,ERROR)

法1:(在网页cloudera manager-flume-配置-类别-日志-Flume Agent 日志目录 找/lvm/log/flume-ng)
http://10.2.5.65:7180/cmf/process/all/logs/search


法2:在命令行
     [root@hadoop6 flume-ng]# cd /lvm/log/flume-ng 
     [root@hadoop6 flume-ng]# cat flume-cmf-flume-AGENT-hadoop6.log |grep sideward_systemdata  (所有日志中筛选需要检查的表的日志 >=IFNO级别)    
     [root@hadoop6 flume-ng]# cat flume-cmf-flume-AGENT-hadoop6.log |grep -i error |grep sideward_systemdata (所有日志中筛选错误的,再在错误中筛选需要检查的表的日志,-i表示不区分大小写)

     [root@hadoop6 flume-ng]# tail -f flume-cmf-flume-AGENT-hadoop6.log (最近10条日志,实时更新)



2.1 查看zookeeper 中保存的kafka的偏移量
[root@hadoop6 data]# zookeeper-client 
[zk: localhost:2181(CONNECTED) 8] get /consumers/BusiSystem2/offsets/BusiSystem/0   //获取所有消费者中消费者组 为BusiSystem2且topic为BusiSystem的第0个分区 的偏移量。
422459392  (这个东西就是当前消费到的位置)
cZxid = 0x1600a8df29
ctime = Thu Jun 22 11:21:56 CST 2017
mZxid = 0x19000c52b3
mtime = Thu May 24 13:53:50 CST 2018
pZxid = 0x1600a8df29
cversion = 0
dataVersion = 163881
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 9
numChildren = 0

[zk: localhost:2181(CONNECTED) 11] ls /consumers/FaultData4/offsets/FaultData
[0]
[zk: localhost:2181(CONNECTED) 12] get /consumers/FaultData4/offsets/FaultData/0
683
cZxid = 0x1d000b0b31
ctime = Thu Aug 09 19:28:25 CST 2018
mZxid = 0x1d000b194b
mtime = Thu Aug 09 21:06:45 CST 2018
pZxid = 0x1d000b0b31
cversion = 0
dataVersion = 1
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 3
numChildren = 0

#将消费者为FaultData4,topic为FaultData的第0个分区的偏移量重新设置为600(本来是683)
[zk: localhost:2181(CONNECTED) 15] set /consumers/FaultData4/offsets/FaultData/0 600
cZxid = 0x1d000b0b31
ctime = Thu Aug 09 19:28:25 CST 2018
mZxid = 0x1d000b83aa
mtime = Fri Aug 10 09:13:05 CST 2018
pZxid = 0x1d000b0b31
cversion = 0
dataVersion = 2
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 3
numChildren = 0

 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值