![](https://img-blog.csdnimg.cn/20201014180756923.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
flume
~Erin
认真你就输了。但是坚持一直认真你就赢了
展开
-
flume使用中遇到的错误
1.Space for commit to queue couldn’t be acquired原因:对于内存通道,当输入过多,而无法全部输出的时候,会导致内存通道崩溃,从而报错。解决办法:设置agent1.channels.<channel_name>.keep-alive = 30...原创 2019-03-27 09:08:35 · 1195 阅读 · 0 评论 -
flume之kafkachannel
1.安装flume2.安装kafka3.测试kafka的topic是否能正常的传递消息4.都准备好之后开始接通kafkachannel,1)kafka的no-sink#定义agent名, source、channel的名称a0.sources = r1a0.channels = c1#具体定义sourcea0.sources.r1.type = execa0.so...原创 2019-05-14 17:20:59 · 2961 阅读 · 0 评论 -
flume之freebsd系统的安装
1.去http://flume.apache.org/网站下载相应的包apache-flume-1.7.0-bin.tar.gz2.用ftp传到相应的机上然后解压在/root/.cshrc里面配置环境变量就可以了3.4.配置conf中的flume-env.sh中的JAVA_HOMEJAVA_HOME=/usr/local/java/jdk1.8.0_1815.然后...原创 2019-05-17 11:08:45 · 127 阅读 · 0 评论 -
flume在非hbase部署的机器上部署,使用aysnHbaseSink
------------------------------------------------------------------------遇到的问题导致我们要部署--------------------------------------------------由于我们使用的flume对接了kafka通道。使用flume的sink充当kafka的消费者消费数据。无意间我们设置成了kafk...原创 2019-11-29 16:27:27 · 172 阅读 · 0 评论 -
hadoop+flume+kafkachannel修改commit cannot be completed due to group reblance的历程
解决办法:增加心跳时间heartbeat.interval.msj解决步骤详解:部署kafkachannel连接flume日志入库之后一直报一个错。前前后后一共改了好多参数session.timeout.ms group coordinator监控consumer的超时时间max.poll.interval.ms 批量处理数据的最大时间max.poll.record...原创 2019-11-14 17:41:31 · 872 阅读 · 1 评论 -
flume之aysnhbase与java的结合
1.flume的sink为aysnhbase的配置# Licensed to the Apache Software Foundation (ASF) under one# or more contributor license agreements. See the NOTICE file# distributed with this work for additional info...原创 2019-09-12 15:19:45 · 130 阅读 · 0 评论 -
flume之多source配置
# Licensed to the Apache Software Foundation (ASF) under one# or more contributor license agreements. See the NOTICE file# distributed with this work for additional information# regarding copyrig...原创 2019-09-12 15:39:51 · 733 阅读 · 0 评论