flink-conf.yaml配置解析

最新推荐文章于 2024-09-25 16:02:50 发布

专注服务端

最新推荐文章于 2024-09-25 16:02:50 发布

阅读量7.2k

点赞数

分类专栏： Flink

本文链接：https://blog.csdn.net/yjh1271845364/article/details/86664674

版权

本文详细解析了Flink的配置文件flink-conf.yaml，包括JobManager和TaskManager的RPC地址与端口、堆内存大小、任务slots数量、默认并行度等核心参数。还涉及高可用性设置，如ZooKeeper集群配置和容错机制。此外，涵盖了检查点和savepoint的存储路径，以及Web Frontend和高级配置选项。

摘要由CSDN通过智能技术生成

################################################################################
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
################################################################################

#==============================================================================
# Common
#==============================================================================

# The external address of the host on which the JobManager runs and can be
# reached by the TaskManagers and any clients which want to connect. This setting
# is only used in Standalone mode and may be overwritten on the JobManager side
# by specifying the --host <hostname> parameter of the bin/jobmanager.sh executable.
# In high availability mode, if you use the bin/start-cluster.sh script and setup
# the conf/masters file, this will be taken care of automatically. Yarn/Mesos
# automatically configure the host name based on the hostname of the node where the
# JobManager runs.

# jobManager 的IP地址
jobmanager.rpc.address: localhost

# The RPC port where the JobManager is reachable.

# jobManager 的端口号
jobmanager.rpc.port: 6123

# The heap size for the JobManager JVM

# jobManager JVM heap 内存大小
jobmanager.heap.size: 1024m

# The heap size for the TaskManager JVM

# taskManager JVM heap 内存大小
taskmanager.heap.size: 1024m

# The number of task slots that each TaskManager offers. Each slot runs one parallel pipeline.

# 每个taskManager提供任务的slots数量大小
taskmanager.numberOfTaskSlots: 1

# The parallelism used for programs that did not specify and other parallelism.

#程序默认的并行计算的个数
parallelism.default: 1

# The default file system scheme and authority.
#
# By default file paths without scheme are interpreted relative to the local
# root file system 'file:///'. Use this to override the default and interpret
#