Flume
文章平均质量分 83
Flume
扛麻袋的少年
得过且过,一天又一天
展开
-
Flume介绍、安装、使用案例、自定义Source/Sink、监控
大数据领域,主要解决的就是3件事情:①海量数据的存储(hdfs) ②海量数据的传输 ③海量数据的计算(MapReduce)。Flume的出现,主要用来解决海量数据的传输问题。原创 2021-06-23 17:35:19 · 686 阅读 · 0 评论 -
Flume案例一:监控 netcat 端口数据案例
Flume案例一:监控 netcat 端口数据案例(选型:netcat source + memory channel + logger sink)原创 2021-06-24 10:51:49 · 2058 阅读 · 1 评论 -
Flume案例二:实时监控单个追加文件(tail -f 日志)(exec source)
Flume案例二:实时监控单个追加文件(tail -f 日志)(选型:exec source + memory channel + logger sink)原创 2021-06-24 13:33:38 · 1375 阅读 · 2 评论 -
Flume案例三:实时监控单个追加文件(tail -f 日志)到HDFS
Flume案例三:实时监控单个追加文件(tail -f 日志)到HDFS(选型:exec source + memory channel + hdfs sink)原创 2021-06-24 14:46:48 · 531 阅读 · 0 评论 -
Flume案例四:实时监控目录下多个新文件(Spooling Directory Source)
四、Flume案例四:实时监控目录下多个新文件(Spooling Directory Source)(选型:spooling directory source + memory channel + hdfs sink)原创 2021-06-24 15:20:40 · 779 阅读 · 0 评论 -
Flume案例五:实时监控多个目录下多个追加文件(Taildir Source)
五、Flume案例五:实时监控多个目录下多个追加文件(Taildir Source)(选型:taildir source + memory channel + hdfs sink)原创 2021-06-24 16:18:06 · 1910 阅读 · 0 评论 -
Flume案例六:复制通道选择器(Replicating Channel Selector)
Flume案例六:复制通道选择器(Replicating Channel Selector)选型: Flume-1:taildir source + memory channel + avro sink + Replicating Channel Selector(复制通道选择器) Flume-1:avro source + memory channel + hdfs sink Flume-1:avro source + memory channel + file roll sink(写入到本地文件原创 2021-06-24 17:29:29 · 662 阅读 · 0 评论 -
Flume案例七:故障转移(Failover Sink Processor)
Flume案例七:故障转移(Failover Sink Processor)选型: Flume-1:netcat source + memory channel + avro sink + Failover Sink Processor(故障转移) Flume-2:avro source + memory channel + logger sink Flume-3:avro source + memory channel + logger sink原创 2021-06-25 16:19:40 · 518 阅读 · 1 评论 -
Flume案例八:负载均衡(Load balancing Sink Processor)
Flume案例八:负载均衡(Load balancing Sink Processor)选型: Flume-1:taildir source + memory channel + avro sink + Load balancing Sink Processor(负载均衡) Flume-2:avro source + memory channel + logger sink Flume-3:avro source + memory channel + logger sink原创 2021-06-25 17:03:07 · 553 阅读 · 6 评论 -
Flume案例九:多数据源聚合
Flume案例九:多数据源聚合选型: Flume-1:taildir source + memory channel + avro sink Flume-2:taildir source + memory channel + avro sink Flume-3:avro source + memory channel + hdfs sink原创 2021-06-28 15:30:17 · 556 阅读 · 2 评论 -
Flume案例十:多路复用通道选择器(Multiplexing Channel Selector)
Flume案例十:多路复用通道选择器(Multiplexing Channel Selector)选型: Flume-1:taildir source + memory channel + avro sink + Multiplexing Channel Selector(多路复用渠道选择器) Flume-2:avro source + memory channel + logger sink Flume-3:avro source + memory channel + logger sink原创 2021-06-28 17:24:16 · 2175 阅读 · 0 评论 -
Flume自定义Source、Sink、数据流监控
参考: 3.6 自定义Source 3.7 自定义Sink 3.8 Flume数据流监控 地址: 阿里云盘:https://www.aliyundrive.com/s/qZqsEN47GZQ 百度网盘:https://pan.baidu.com/s/1j4t8w0rtZ5gVNdy8Fqwo2g 提取码:cobq原创 2021-08-05 11:42:03 · 210 阅读 · 0 评论