Concepts:Flink Architecture

Flink Architecture Flink架构

Flink is a distributed system and requires effective allocation and management of compute resources in order to execute streaming applications. It integrates with all common cluster resource managers such as Hadoop YARN and Kubernetes, but can also be set up to run as a standalone cluster or even as a library.
Flink是一个分布式系统,需要有效分配和管理计算资源才能执行流式应用程序。它与所有常见的集群资源管理器(如Hadoop YARN和Kubernetes)集成,但也可以作为独立集群运行,甚至作为库运行。

This section contains an overview of Flink’s architecture and describes how its main components interact to execute applications and recover from failures.
本节概述Flink的架构,并描述其主要组件如何交互以执行应用程序和从故障中恢复。

Anatomy of a Flink Cluster Flink集群剖析

The Flink runtime consists of two types of processes: a JobManager and one or more TaskManagers.
Flink runtime由两种类型的进程组成:一个JobManager和一个或多个TaskManager。
在这里插入图片描述

The Client is not part of the runtime and program execution, but is used to prepare and send a dataflow to the JobManager. After that, the client can disconnect (detached mode), or stay connected to receive progress reports (attached mode). The client runs either as part of the Java/Scala program that triggers the execution, or in the command line process ./bin/flink run …
客户端不是runtime和程序执行的一部分,而是用于准备dataflow并将其发送到JobManager。之后,客户端可以断开连接(分离模式),或保持连接以接收进度报告(连接模式)。客户端可以作为触发执行的Java/Scala程序的一部分运行,也可以在命令行进程中运行 ./bin/flink run …

The JobManager and TaskManagers can be started in various ways: directly on the machines as a standalone cluster, in containers, or managed by resource frameworks like YARN. TaskManagers connect to JobManagers, announcing themselves as available, and are assigned work.
JobManager和TaskManager可以以多种方式启动:作为独立集群直接在机器上启动,在容器中启动,或者由诸如YARN之类的资源框架管理。TaskManager连接到JobManager,宣布自己可用,并被分配工作。

JobManager

The JobManager has a number of responsibilities related to coordinating the distributed execution of Flink Applications: it decides when to schedule the next task (or set of tasks), reacts to finished tasks or execution failures, coordinates checkpoints, and coordinates recovery on failures, among others. This process consists of three different components:
JobManager有许多与协调Flink应用程序的分布式执行相关的职责:它决定何时调度下一个任务(或一组任务),对完成的任务或执行失败做出反应,协调检查点,并协调故障恢复,等等。该过程由三个不同的部分组成:

  • ResourceManager
    The ResourceManager is responsible for resource de-/allocation and provisioning in a Flink cluster — it manages task slots, which are the unit of resource scheduling in a Flink cluster (see TaskManagers). Flink implements multiple ResourceManagers for different environments and resource providers such as YARN, Kubernetes and standalone deployments. In a standalone setup, the ResourceManager c

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值