flume笔记(四) Channal

版权声明:本文为博主原创文章,遵循 CC 4.0 BY-SA 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://blog.csdn.net/ting_163/article/details/84901023

flume原理

在这里插入图片描述

memory channel(内存通道):

  1. 配置memory.conf:
a1.sources = r1
a1.channels = c1
a1.sinks = k1

a1.channels.c1.type = memory
a1.channels.c1.capacity = 10000
a1.channels.c1.transactionCapacity = 10000
a1.channels.c1.byteCapacityBufferPercentage = 20
a1.channels.c1.byteCapacity = 800000

a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
  1. 启动flume:
    $> bin/flume-ng agent -f conf/memory.conf -n al -Dflume.root.logger=INFO,console

file channel(文件通道):

  1. 配置file.conf
a1.sources = r1
a1.channels = c1
a1.sinks = k1


a1.channels.c1.type = file
a1.channels.c1.checkpointDir = /home/ubuntu/data/flume/checkpoint
a1.channels.c1.dataDirs = /home/ubuntu/data/flume/data

a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
  1. 启动flume:
    $> bin/flume-ng agent -f conf/file.conf -n al -Dflume.root.logger=INFO,console

kafka channel(kafka通道):

展开阅读全文

没有更多推荐了,返回首页