Flume 学习笔记 一 ： Flume User Guide

1.  Architecture 总体概述

Flume’s architecture is simple, robust, and flexible. The main abstraction in Flume is a stream-oriented data flow . A data flow describes the way a single stream of data is transferred and processed from its point of generation to its eventual destination. Data flows are composed of logical nodes that can transform or aggregate the events they receive. Logical nodes are wired together in chains to form a data flow. The way in which they are wired is called the logical node’s configuration .

Controlling all this is the Flume Master , which is a separate service with knowledge of all the physical and logical nodes in a Flume installation. The Master assigns configurations to logical nodes, and is responsible for communicating configuration updates by the user to all logical nodes. In turn, the logical nodes periodically contact the master so they can share monitoring information and check for updates to their configuration.

 

三层结构：