Apache Seatunnel部署（apache-seatunnel-2.3.3-bin.tar.gz + apache-seatunnel-web-1.0.0-bin.tar.gz）

最新推荐文章于 2024-06-17 14:48:07 发布
最新推荐文章于 2024-06-17 14:48:07 发布
阅读量2.8k 收藏 38
点赞数 25
分类专栏： ETL&ELT 文章标签： seatunnel 核心引擎 web界面
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/qq_36434219/article/details/134857560
版权

目录

🍑一、概要

安装包及源码包下载地址
在这里插入图片描述
在这里插入图片描述

Apache SeaTunnel 是一个分布式、高性能、易扩展的数据集成平台，用于实时和离线数据处理。它支持多种数据源之间的数据迁移和转换。其中，apache-seatunnel-web-1.0.0-bin.tar.gz 和 apache-seatunnel-2.3.3-bin.tar.gz 分别代表了 Apache SeaTunnel Web 界面的安装包和 Apache SeaTunnel 的核心引擎的安装包。

  • apache-seatunnel-web-1.0.0-bin.tar.gz：这是 Apache SeaTunnel Web 的安装包，提供了用户友好的 Web 界面，使得用户可以更方便地管理和监控 SeaTunnel 任务，包括创建、编辑和运行作业等操作。通过这个界面，用户可以无需直接接触命令行就能进行配置和管理。
  • apache-seatunnel-2.3.3-bin.tar.gz：这是 Apache SeaTunnel 核心引擎的二进制发行版，包含了执行数据抽取、转换和加载的核心代码库。它是实际执行数据处理任务的部分，可以与各种数据源和目标（如数据库、文件系统、消息队列等）进行交互。

在部署 Apache SeaTunnel 时，通常需要先安装并配置好核心引擎，然后根据需要选择是否使用 Web 界面进行管理。如果你只需要在命令行下运行和管理任务，那么只需要安装核心引擎即可。如果你想拥有一个图形化的任务管理界面，那么就需要同时安装 Apache SeaTunnel Web。

🍑二、核心引擎部署

🍊 上传解压安装包： tar xf apache-seatunnel-2.3.3-bin.tar.gz
在这里插入图片描述

🍊 自动安装

🍓自动安装插件（全量安装，很慢，一直到下班，一个半小时都没结束，第二天早上来显示下载结束）

[opensource@bigdata02 bin]$ pwd
/home/opensource/app/apache-seatunnel-2.3.3/bin
[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ ./install-plugin.sh

在这里插入图片描述

🍓 验证服务

🍒执行系统自带的任务进行验证： sh seatunnel.sh --config ../config/v2.batch.config.template -m local
在这里插入图片描述

[opensource@bigdata02 bin]$ sh seatunnel.sh --config ../config/v2.batch.config.template -m local 
十二月 08, 2023 9:09:51 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:09:51 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:09:51 上午 org.apache.seatunnel.engine.common.config.SeaTunnelConfig
信息: seatunnel.home is /home/opensource/app/apache-seatunnel-2.3.3/bin
十二月 08, 2023 9:09:51 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:09:51 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
2023-12-08 09:09:51,483 WARN  com.hazelcast.instance.AddressPicker - [LOCAL] [seatunnel-233929] [5.1] You configured your member address as host name. Please be aware of that your dns can be spoofed. Make sure that your dns configurations are correct.
2023-12-08 09:09:51,487 INFO  com.hazelcast.instance.AddressPicker - [LOCAL] [seatunnel-233929] [5.1] Resolving domain name 'localhost' to address(es): [127.0.0.1]
2023-12-08 09:09:51,487 INFO  com.hazelcast.instance.AddressPicker - [LOCAL] [seatunnel-233929] [5.1] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [localhost/127.0.0.1]
2023-12-08 09:09:51,509 INFO  org.apache.seatunnel.engine.server.SeaTunnelServer - SeaTunnel server start...
2023-12-08 09:09:51,510 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] Based on Hazelcast IMDG version: 5.1.0 (20220228 - 21f20e7)
2023-12-08 09:09:51,510 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] Cluster name: seatunnel-233929
2023-12-08 09:09:51,510 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] 
                                                         
 _____               _____                             _ 
/  ___|             |_   _|                           | |
\ `--.   ___   __ _   | |   _   _  _ __   _ __    ___ | |
 `--. \ / _ \ / _` |  | |  | | | || '_ \ | '_ \  / _ \| |
/\__/ /|  __/| (_| |  | |  | |_| || | | || | | ||  __/| |
\____/  \___| \__,_|  \_/   \__,_||_| |_||_| |_| \___||_|
                                                         

2023-12-08 09:09:51,511 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] Copyright © 2021-2022 The Apache Software Foundation. Apache SeaTunnel, SeaTunnel, and its feather logo are trademarks of The Apache Software Foundation.
2023-12-08 09:09:51,511 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following: 
  - Change member config using Java API: config.setIntegrityCheckerEnabled(true);
  - Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
  - Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2023-12-08 09:09:51,513 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
  - Change member config using Java API: config.getJetConfig().setEnabled(true)
  - Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
  - Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2023-12-08 09:09:51,867 INFO  com.hazelcast.system.security - [localhost]:5801 [seatunnel-233929] [5.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security  or use -Dhazelcast.security.recommendations system property to see  security recommendations and the status of current config.
2023-12-08 09:09:51,919 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-233929] [5.1] Using TCP/IP discovery
2023-12-08 09:09:51,920 WARN  com.hazelcast.cp.CPSubsystem - [localhost]:5801 [seatunnel-233929] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
十二月 08, 2023 9:09:52 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:09:52 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:09:52 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:09:52 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
2023-12-08 09:09:52,071 WARN  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] The Node is not ready yet, Node state STARTING,looking forward to the next scheduling
2023-12-08 09:09:52,072 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] Created new BusWork : 1217639236
2023-12-08 09:09:52,082 WARN  org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2023-12-08 09:09:52,084 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2023-12-08 09:09:52,121 INFO  com.hazelcast.internal.diagnostics.Diagnostics - [localhost]:5801 [seatunnel-233929] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2023-12-08 09:09:52,128 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5801 is STARTING
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5802 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5817 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5819 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5823 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5804 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5821 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5813 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5825 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5815 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5808 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5806 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5829 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5810 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5827 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5822 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5820 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5814 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5812 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5826 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5805 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5807 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5818 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5816 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5830 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5809 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5811 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5824 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5828 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5803 is added to the blacklist.
2023-12-08 09:09:53,162 INFO  com.hazelcast.internal.cluster.ClusterService - [localhost]:5801 [seatunnel-233929] [5.1] 

Members {size:1, ver:1} [
        Member [localhost]:5801 - 87c80e4a-80c5-4b82-b577-cb0649025b5e this
]

2023-12-08 09:09:53,172 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5801 is STARTED
十二月 08, 2023 9:09:53 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
十二月 08, 2023 9:09:53 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast-client.yaml
2023-12-08 09:09:53,269 INFO  com.hazelcast.client.impl.spi.ClientInvocationService - hz.client_1 [seatunnel-233929] [5.1] Running with 2 response threads, dynamic=true
2023-12-08 09:09:53,279 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2023-12-08 09:09:53,279 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2023-12-08 09:09:53,285 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-233929] [5.1] Trying to connect to cluster: seatunnel-233929
2023-12-08 09:09:53,286 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-233929] [5.1] Trying to connect to [localhost]:5801
2023-12-08 09:09:53,299 INFO  com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask - [localhost]:5801 [seatunnel-233929] [5.1] Received auth from Connection[id=1, /127.0.0.1:5801->/127.0.0.1:55059, qualifier=null, endpoint=[127.0.0.1]:55059, remoteUuid=535a4a1b-58b7-40a4-8909-218d69fedfe5, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 535a4a1b-58b7-40a4-8909-218d69fedfe5, client name: hz.client_1, client version: 5.1
2023-12-08 09:09:53,302 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2023-12-08 09:09:53,302 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-233929] [5.1] Authenticated with server [localhost]:5801:87c80e4a-80c5-4b82-b577-cb0649025b5e, server version: 5.1, local address: /127.0.0.1:55059
2023-12-08 09:09:53,303 INFO  com.hazelcast.internal.diagnostics.Diagnostics - hz.client_1 [seatunnel-233929] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2023-12-08 09:09:53,308 INFO  com.hazelcast.client.impl.spi.ClientClusterService - hz.client_1 [seatunnel-233929] [5.1] 

Members [1] {
        Member [localhost]:5801 - 87c80e4a-80c5-4b82-b577-cb0649025b5e
}

2023-12-08 09:09:53,329 INFO  com.hazelcast.client.impl.statistics.ClientStatisticsService - Client statistics is enabled with period 5 seconds.
2023-12-08 09:09:53,453 INFO  org.apache.seatunnel.engine.client.job.JobExecutionEnvironment - add common jar in plugins :[]
2023-12-08 09:09:53,466 INFO  org.apache.seatunnel.core.starter.utils.ConfigBuilder - Loading config file from path: ../config/v2.batch.config.template
2023-12-08 09:09:53,504 INFO  org.apache.seatunnel.core.starter.utils.ConfigShadeUtils - Load config shade spi: [base64]
2023-12-08 09:09:53,540 INFO  org.apache.seatunnel.core.starter.utils.ConfigBuilder - Parsed config file: {
    "env" : {
        "execution.parallelism" : 2,
        "job.mode" : "BATCH",
        "checkpoint.interval" : 10000
    },
    "source" : [
        {
            "schema" : {
                "fields" : {
                    "name" : "string",
                    "age" : "int"
                }
            },
            "row.num" : 16,
            "parallelism" : 2,
            "result_table_name" : "fake",
            "plugin_name" : "FakeSource"
        }
    ],
    "sink" : [
        {
            "plugin_name" : "Console"
        }
    ]
}

2023-12-08 09:09:53,560 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,561 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,564 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSink Plugin from /home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel
2023-12-08 09:09:53,568 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: FakeSource at: file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-fake-2.3.3.jar
2023-12-08 09:09:53,568 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: Console at: file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-console-2.3.3.jar
2023-12-08 09:09:53,571 INFO  org.apache.seatunnel.engine.core.parse.ConfigParserUtil - Currently, incorrect configuration of source_table_name and result_table_name options don't affect job running. In the future we will ban incorrect configurations.
2023-12-08 09:09:53,571 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,571 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,571 WARN  org.apache.seatunnel.engine.core.parse.ConfigParserUtil - This configuration is not recommended. A source/transform(FakeSource) is configured with 'result_table_name' option value of 'fake', but subsequent transform/sink(Console) is not configured with 'source_table_name' option.
2023-12-08 09:09:53,572 INFO  org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser - start generating all sources.
2023-12-08 09:09:53,572 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,581 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSource Plugin from /home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel
2023-12-08 09:09:53,585 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: FakeSource at: file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-fake-2.3.3.jar
2023-12-08 09:09:53,586 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='FakeSource'} from classpath
2023-12-08 09:09:53,603 INFO  org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser - start generating all transforms.
2023-12-08 09:09:53,603 INFO  org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser - start generating all sinks.
2023-12-08 09:09:53,603 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,607 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,636 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Start submit job, job id: 785312093409640449, with plugin jar [file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-fake-2.3.3.jar, file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-console-2.3.3.jar]
2023-12-08 09:09:53,640 WARN  org.apache.seatunnel.engine.server.SeaTunnelServer - This is master node, waiting the coordinator service init finished
2023-12-08 09:09:54,086 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:09:54,114 INFO  com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel-233929] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:09:54,141 WARN  org.apache.seatunnel.engine.server.SeaTunnelServer - This is master node, waiting the coordinator service init finished
2023-12-08 09:09:54,646 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:09:54,646 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker... 
2023-12-08 09:09:54,646 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:09:54,648 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=1029177344}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=1029177344}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:09:54,662 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Init JobMaster for Job SeaTunnel_Job (785312093409640449) 
2023-12-08 09:09:54,662 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job SeaTunnel_Job (785312093409640449) needed jar urls [file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-fake-2.3.3.jar, file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-console-2.3.3.jar]
2023-12-08 09:09:54,874 WARN  org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2023-12-08 09:09:54,976 INFO  org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage - Path /tmp/seatunnel/checkpoint_snapshot/785312093409640449 is not a directory
2023-12-08 09:09:54,976 INFO  org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage - No checkpoint found for job, job id is: 785312093409640449
2023-12-08 09:09:54,978 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] is in state CREATED when init state future
2023-12-08 09:09:54,979 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] is in state CREATED when init state future
2023-12-08 09:09:54,979 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] is in state CREATED when init state future
2023-12-08 09:09:54,981 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Submit job finished, job id: 785312093409640449, job name: SeaTunnel
2023-12-08 09:09:54,982 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job Job SeaTunnel_Job (785312093409640449) waiting for scheduler finished
2023-12-08 09:09:54,984 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (785312093409640449) turn from state CREATED to SCHEDULED.
2023-12-08 09:09:54,987 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] turn from state CREATED to SCHEDULED.
2023-12-08 09:09:54,989 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] state from CREATED to SCHEDULED
2023-12-08 09:09:54,990 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] turn from state CREATED to SCHEDULED.
2023-12-08 09:09:54,994 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] state from CREATED to SCHEDULED
2023-12-08 09:09:54,994 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] turn from state CREATED to SCHEDULED.
2023-12-08 09:09:54,995 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] state from CREATED to SCHEDULED
2023-12-08 09:09:54,996 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] turn from state CREATED to SCHEDULED.
2023-12-08 09:09:55,003 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] turn from state SCHEDULED to DEPLOYING.
2023-12-08 09:09:55,005 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] state from SCHEDULED to DEPLOYING
2023-12-08 09:09:55,006 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] turn from state SCHEDULED to DEPLOYING.
2023-12-08 09:09:55,006 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] state from SCHEDULED to DEPLOYING
2023-12-08 09:09:55,007 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] turn from state SCHEDULED to DEPLOYING.
2023-12-08 09:09:55,008 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] state from SCHEDULED to DEPLOYING
2023-12-08 09:09:55,009 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] turn from state SCHEDULED to DEPLOYING.
2023-12-08 09:09:55,009 WARN  org.apache.seatunnel.engine.client.job.JobMetricsRunner - Failed to get job metrics summary, it maybe first-run
2023-12-08 09:09:55,016 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] received deploying task executionId [785312099889840129]
2023-12-08 09:09:55,025 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] received deploying task executionId [785312099889971201]
2023-12-08 09:09:55,025 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] received deploying task executionId [785312099889905665]
2023-12-08 09:09:55,034 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1}, executionId [785312099889840129]
2023-12-08 09:09:55,035 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1} init success
2023-12-08 09:09:55,040 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1} success
2023-12-08 09:09:55,040 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] state from DEPLOYING to RUNNING
2023-12-08 09:09:55,042 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - starting seatunnel source split enumerator task, source name: pipeline-1 [Source[0]-FakeSource-fake]
2023-12-08 09:09:55,042 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] turn from state DEPLOYING to RUNNING.
2023-12-08 09:09:55,047 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}, executionId [785312099889971201]
2023-12-08 09:09:55,048 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001} init success
2023-12-08 09:09:55,048 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}, executionId [785312099889905665]
2023-12-08 09:09:55,049 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001} success
2023-12-08 09:09:55,050 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] state from DEPLOYING to RUNNING
2023-12-08 09:09:55,051 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000} init success
2023-12-08 09:09:55,052 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] turn from state DEPLOYING to RUNNING.
2023-12-08 09:09:55,052 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000} success
2023-12-08 09:09:55,053 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] state from DEPLOYING to RUNNING
2023-12-08 09:09:55,054 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] turn from state DEPLOYING to RUNNING.
2023-12-08 09:09:55,054 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] turn from state DEPLOYING to RUNNING.
2023-12-08 09:09:55,054 INFO  org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask - starting seatunnel transform task, index 0
2023-12-08 09:09:55,054 INFO  org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask - starting seatunnel transform task, index 1
2023-12-08 09:09:55,060 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (785312093409640449) turn from state SCHEDULED to RUNNING.
2023-12-08 09:09:55,061 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job SeaTunnel_Job (785312093409640449) scheduler finished
2023-12-08 09:09:55,064 INFO  org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask - starting seatunnel source task, index 0
2023-12-08 09:09:55,064 INFO  org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask - starting seatunnel source task, index 1
2023-12-08 09:09:55,068 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - output rowType: name<STRING>, age<INT>
2023-12-08 09:09:55,068 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - output rowType: name<STRING>, age<INT>
2023-12-08 09:09:55,166 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}, taskID=40001, index=1}
2023-12-08 09:09:55,166 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}
2023-12-08 09:09:55,245 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received enough reader, starting enumerator...
2023-12-08 09:09:55,245 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Starting to calculate splits.
2023-12-08 09:09:55,246 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Assigned [FakeSourceSplit(splitId=1, rowNum=16), FakeSourceSplit(splitId=0, rowNum=16)] to 2 readers.
2023-12-08 09:09:55,246 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Calculated splits successfully, the size of splits is 2.
2023-12-08 09:09:55,247 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Assigning splits to readers 0 [FakeSourceSplit(splitId=0, rowNum=16)]
2023-12-08 09:09:55,251 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Assigning splits to readers 1 [FakeSourceSplit(splitId=1, rowNum=16)]
2023-12-08 09:09:55,269 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceReader - 16 rows of data have been generated in split(0). Generation time: 1701997795267
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceReader - Closed the bounded fake source
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceReader - 16 rows of data have been generated in split(1). Generation time: 1701997795266
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceReader - Closed the bounded fake source
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=1:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : kkTzr, 348993738
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=1:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : cHRlJ, 141341247
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=2:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : JoqKS, 949667288
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=2:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : wuRPB, 739004063
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=3:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : vzyWZ, 1185127730
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=3:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : MzsIj, 255990586
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=4:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : pKInq, 1307779433
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=4:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : JmaUB, 255860364
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=5:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : MYeRA, 1997823969
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=5:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : uVILC, 995215612
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=6:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : DlVbk, 731828921
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=7:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : inYyz, 1847828742
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=6:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : cyjxD, 1832606505
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=7:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : eMzXN, 802135106
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=8:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : kPUMq, 1916092538
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=8:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : aySgR, 1299495336
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=9:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : vFyQU, 1895773363
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=9:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : pjYqn, 1385074915
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=10:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : dQmgk, 1234357286
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=10:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : zASIv, 1497863961
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=11:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : zQuTA, 903385081
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=11:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : IVXOt, 870847193
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=12:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : jrLTl, 705955925
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=13:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : YflRj, 293817756
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=12:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : DbvZU, 1486490099
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=14:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : SeaoD, 1066109912
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=13:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : VsbMy, 910565833
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=15:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : vkEhB, 403795863
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=14:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : fegPV, 165600914
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=16:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : rXrEp, 1087983282
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=15:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : DokOA, 1281308771
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=16:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : VjDeD, 1091404164
2023-12-08 09:09:55,356 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - wait checkpoint completed: 9223372036854775807
2023-12-08 09:09:55,411 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - pending checkpoint(9223372036854775807/1@785312093409640449) notify finished!
2023-12-08 09:09:55,411 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start notify checkpoint completed, checkpoint:org.apache.seatunnel.engine.server.checkpoint.CompletedCheckpoint@4d1c18a2
2023-12-08 09:09:55,416 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start clean pending checkpoint cause CheckpointCoordinator completed.
2023-12-08 09:09:55,417 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Turn checkpoint_state_785312093409640449_1 state from null to FINISHED
2023-12-08 09:09:55,453 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 20000, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1}
2023-12-08 09:09:55,453 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] Task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1} complete with state FINISHED
2023-12-08 09:09:55,454 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] Received task end from execution TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1}, state FINISHED
2023-12-08 09:09:55,455 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] turn to end state FINISHED.
2023-12-08 09:09:55,455 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] end with state FINISHED
2023-12-08 09:09:55,464 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 50001, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}
2023-12-08 09:09:55,464 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 50000, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}
2023-12-08 09:09:56,271 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 40001, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}
2023-12-08 09:09:56,271 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 40000, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}
2023-12-08 09:09:56,272 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] Task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000} complete with state FINISHED
2023-12-08 09:09:56,272 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] Task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001} complete with state FINISHED
2023-12-08 09:09:56,272 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] Received task end from execution TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}, state FINISHED
2023-12-08 09:09:56,272 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] Received task end from execution TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}, state FINISHED
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] turn to end state FINISHED.
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] turn to end state FINISHED.
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] end with state FINISHED
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] end with state FINISHED
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] end with state FINISHED
2023-12-08 09:09:56,310 INFO  org.apache.seatunnel.engine.server.master.JobMaster - release the pipeline Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] resource
2023-12-08 09:09:56,311 INFO  org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 785312093409640449, slot: SlotProfile{worker=[localhost]:5801, slotID=1, ownerJobID=785312093409640449, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='fa7bad5a-cd2c-4d97-8368-88027b3b49b6'}
2023-12-08 09:09:56,311 INFO  org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 785312093409640449, slot: SlotProfile{worker=[localhost]:5801, slotID=2, ownerJobID=785312093409640449, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='fa7bad5a-cd2c-4d97-8368-88027b3b49b6'}
2023-12-08 09:09:56,312 INFO  org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 785312093409640449, slot: SlotProfile{worker=[localhost]:5801, slotID=3, ownerJobID=785312093409640449, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='fa7bad5a-cd2c-4d97-8368-88027b3b49b6'}
2023-12-08 09:09:56,314 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] turn to end state FINISHED.
2023-12-08 09:09:56,315 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (785312093409640449) end with state FINISHED
2023-12-08 09:09:56,328 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Job (785312093409640449) end with state FINISHED
2023-12-08 09:09:56,353 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - 
***********************************************
           Job Statistic Information
***********************************************
Start Time                : 2023-12-08 09:09:53
End Time                  : 2023-12-08 09:09:56
Total Time(s)             :                   2
Total Read Count          :                  32
Total Write Count         :                  32
Total Failed Count        :                   0
***********************************************

2023-12-08 09:09:56,354 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2023-12-08 09:09:56,357 INFO  com.hazelcast.internal.server.tcp.TcpServerConnection - [localhost]:5801 [seatunnel-233929] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:55059, qualifier=null, endpoint=[127.0.0.1]:55059, remoteUuid=535a4a1b-58b7-40a4-8909-218d69fedfe5, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2023-12-08 09:09:56,357 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-233929] [5.1] Removed connection to endpoint: [localhost]:5801:87c80e4a-80c5-4b82-b577-cb0649025b5e, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:55059->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2023-12-08 09:09:56.351, lastWriteTime=2023-12-08 09:09:56.328, closedTime=2023-12-08 09:09:56.355, connected server version=5.1}
2023-12-08 09:09:56,358 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2023-12-08 09:09:56,358 INFO  com.hazelcast.client.impl.ClientEndpointManager - [localhost]:5801 [seatunnel-233929] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:55059, qualifier=null, endpoint=[127.0.0.1]:55059, remoteUuid=535a4a1b-58b7-40a4-8909-218d69fedfe5, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=535a4a1b-58b7-40a4-8909-218d69fedfe5, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1701997793296, latest clientAttributes=lastStatisticsCollectionTime=1701997793329,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1701997793287,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=24435175424,os.freePhysicalMemorySize=50645405696,os.freeSwapSpaceSize=0,os.maxFileDescriptorCount=65536,os.openFileDescriptorCount=58,os.processCpuTime=9670000000,os.systemLoadAverage=2.31,os.totalPhysicalMemorySize=540422402048,os.totalSwapSpaceSize=0,runtime.availableProcessors=104,runtime.freeMemory=959249984,runtime.maxMemory=1029177344,runtime.totalMemory=1029177344,runtime.uptime=2770,runtime.usedMemory=69927360, labels=[]}
2023-12-08 09:09:56,360 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2023-12-08 09:09:56,360 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client......
2023-12-08 09:09:56,360 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5801 is SHUTTING_DOWN
2023-12-08 09:09:56,363 INFO  com.hazelcast.internal.partition.impl.MigrationManager - [localhost]:5801 [seatunnel-233929] [5.1] Shutdown request of Member [localhost]:5801 - 87c80e4a-80c5-4b82-b577-cb0649025b5e this is handled
2023-12-08 09:09:56,367 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-233929] [5.1] Shutting down connection manager...
2023-12-08 09:09:56,369 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-233929] [5.1] Shutting down node engine...
2023-12-08 09:09:58,473 INFO  com.hazelcast.instance.impl.NodeExtension - [localhost]:5801 [seatunnel-233929] [5.1] Destroying node NodeExtension.
2023-12-08 09:09:58,474 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-233929] [5.1] Hazelcast Shutdown is completed in 2111 ms.
2023-12-08 09:09:58,474 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5801 is SHUTDOWN
2023-12-08 09:09:58,474 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed HazelcastInstance ......
2023-12-08 09:09:58,474 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed metrics executor service ......
2023-12-08 09:09:58,475 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - run shutdown hook because get close signal
[opensource@bigdata02 bin]$

🍓 启动服务

🍒启动服务并且后台运行，用于web端连接：nohup sh seatunnel-cluster.sh 2>&1 &
在这里插入图片描述


[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &    
[1] 93523
[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"

[opensource@bigdata02 bin]$ ls
install-plugin.sh  seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh  stop-seatunnel-cluster.sh
nohup.out          seatunnel.sh          start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
[opensource@bigdata02 bin]$ tail -f nohup.out 
十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
^C
[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log 
        Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this
]

2023-12-08 09:30:59,930 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED
2023-12-08 09:31:01,865 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:31:02,061 INFO  com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker... 
2023-12-08 09:31:04,864 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:31:04,865 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:31:59,863 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2023-12-08 09:31:59,867 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
                Job info detail
***********************************************
createdJobCount           :                   0
scheduledJobCount         :                   0
runningJobCount           :                   0
failingJobCount           :                   0
failedJobCount            :                   0
cancellingJobCount        :                   0
canceledJobCount          :                   0
finishedJobCount          :                   0
restartingJobCount        :                   0
suspendedJobCount         :                   0
reconcilingJobCount       :                   0
***********************************************

🍊 手动安装

  • 不想全部安装，修改配置：vi config/plugin_config （具体保留哪些就不清楚了，只能说官方提供的安装包不好，无法直接启动，必须要先安装插件，但是这么多插件下载又很慢、也没必要）
    在这里插入图片描述
  • 无法在线安装，只能手动了
    maven库下载地址
    在这里插入图片描述
    对于自动下载下来的jar包，已经全部打包上传到CSDN,大家可以免费下载，直接放到connectors/seatunnel
    在这里插入图片描述
    下载地址
    在这里插入图片描述
    CSDN资源大小不允许超过1000M,这里采用.tar.xz格式（解压会有些慢，小十分钟吧）
    • 三种压缩命令：
      tar -cf seatunnel_connectors_2.3.3.tar seatunnel
tar -czf seatunnel_connectors_2.3.3.tar.gz seatunnel
tar -cJf seatunnel_connectors_2.3.3.tar.xz seatunnel
    • 对应解压命令：
      tar -xf seatunnel_connectors_2.3.3.tar 
tar -xzf seatunnel_connectors_2.3.3.tar.gz 
tar -xJf seatunnel_connectors_2.3.3.tar.xz
      在这里插入图片描述

🍑三、Web界面部署

🍊 上传解压web部署包：tar xf apache-seatunnel-web-1.0.0-bin.tar.gz
在这里插入图片描述

🍊 初始化数据库

🍓 初始化SQL脚本script/seatunnel_server_mysql.sql
在这里插入图片描述
在这里插入图片描述

SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for role
-- ----------------------------
DROP TABLE IF EXISTS `role`;
CREATE TABLE `role`  (
  `id` int(20) NOT NULL AUTO_INCREMENT,
  `type` int(2) NOT NULL,
  `role_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Records of role
-- ----------------------------
INSERT INTO `role`(`type`,`role_name`,`description`) values (0, 'ADMIN_ROLE', 'Admin User');
INSERT INTO `role`(`type`,`role_name`,`description`) values (1, 'NORMAL_ROLE', 'Normal User');

-- ----------------------------
-- Table structure for role_user_relation
-- ----------------------------
DROP TABLE IF EXISTS `role_user_relation`;
CREATE TABLE `role_user_relation`  (
  `id` int(20) NOT NULL AUTO_INCREMENT,
  `role_id` int(20) NOT NULL,
  `user_id` int(20) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_datasource
-- ----------------------------
DROP TABLE IF EXISTS `t_st_datasource`;
CREATE TABLE `t_st_datasource`  (
  `id` bigint(20) NOT NULL,
  `datasource_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `plugin_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `plugin_version` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT '1.0.0',
  `datasource_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `t_st_datasource_datasource_name_uindex`(`datasource_name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_definition
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_definition`;
CREATE TABLE `t_st_job_definition`  (
  `id` bigint(20) NOT NULL,
  `name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `name`(`name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_instance
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance`;
CREATE TABLE `t_st_job_instance`  (
  `id` bigint(20) NOT NULL,
  `job_define_id` bigint(20) NOT NULL,
  `job_status` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `job_config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `job_engine_id` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(20) NOT NULL,
  `update_user_id` int(20) NULL DEFAULT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  `end_time` timestamp(3) NULL DEFAULT NULL,
  `job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_instance_history
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance_history`;
CREATE TABLE `t_st_job_instance_history`  (
  `id` bigint(20) NOT NULL,
  `dag` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_line
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_line`;
CREATE TABLE `t_st_job_line`  (
  `id` bigint(20) NOT NULL,
  `version_id` bigint(20) NOT NULL,
  `input_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `target_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `job_line_version_index`(`version_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_metrics
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_metrics`;
CREATE TABLE `t_st_job_metrics`  (
  `id` bigint(20) NOT NULL,
  `job_instance_id` bigint(20) NOT NULL,
  `pipeline_id` int(20) NOT NULL,
  `read_row_count` bigint(20) NOT NULL,
  `write_row_count` bigint(20) NOT NULL,
  `source_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `sink_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `read_qps` bigint(20) NULL DEFAULT NULL,
  `write_qps` bigint(20) NULL DEFAULT NULL,
  `record_delay` bigint(20) NULL DEFAULT NULL,
  `status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(20) NOT NULL,
  `update_user_id` int(20) NULL DEFAULT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_task
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_task`;
CREATE TABLE `t_st_job_task`  (
  `id` bigint(20) NOT NULL,
  `version_id` bigint(20) NOT NULL,
  `plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `transform_options` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `output_schema` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `connector_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `datasource_id` bigint(20) NULL DEFAULT NULL,
  `datasource_option` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `select_table_fields` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `scene_mode` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `job_task_plugin_id_index`(`plugin_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_version
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_version`;
CREATE TABLE `t_st_job_version`  (
  `id` bigint(20) NOT NULL,
  `job_id` bigint(20) NOT NULL,
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `job_mode` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `env` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_virtual_table
-- ----------------------------
DROP TABLE IF EXISTS `t_st_virtual_table`;
CREATE TABLE `t_st_virtual_table`  (
  `id` bigint(20) NOT NULL,
  `datasource_id` bigint(20) NOT NULL,
  `virtual_database_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `virtual_table_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `table_fields` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `virtual_table_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for user
-- ----------------------------
DROP TABLE IF EXISTS `user`;
CREATE TABLE `user`  (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `password` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `status` tinyint(4) NOT NULL,
  `type` tinyint(4) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 2 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for user_login_log
-- ----------------------------
DROP TABLE IF EXISTS `user_login_log`;
CREATE TABLE `user_login_log`  (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `token` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `token_status` tinyint(1) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 106 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Records of user_login_log
-- ----------------------------

INSERT INTO `user`(`username`,`password`,`status`,`type`) values ('admin', '7f97da8846fed829bb8d1fd9f8030f3b', 0, 0);

SET FOREIGN_KEY_CHECKS = 1;

🍊 配置文件修改

🍓 配置数据库信息： vi conf/application.yml
在这里插入图片描述
🍓 复制引擎服务中配置文件到web配置目录下面

[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/config/hazelcast-client.yaml apache-seatunnel-web-1.0.0-bin/conf/
[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/connectors/plugin-mapping.properties apache-seatunnel-web-1.0.0-bin/conf/

在这里插入图片描述

🍊 配置下载数据源jar包

🍓 选择的mysql作为元数据库，对应的驱动包mysql-connector-java-8.0.25.jar放到libs下
在这里插入图片描述
🍓 下载其它数据源资源包,首先下载源码，然后执行下载脚本download_datasource.sh(本地安装过git)

  • 🍒源码地址
    在这里插入图片描述
  • 🍒修改download_datasource.sh相关配置
    在这里插入图片描述
    在这里插入图片描述
  • 🍒执行download_datasource.sh
    在这里插入图片描述
  • 🍒开始下载jar包
    在这里插入图片描述
  • 🍒下载完成
    在这里插入图片描述
  • 🍒上传到libs目录
    在这里插入图片描述

🍊 启动服务

sh bin/seatunnel-backend-daemon.sh start

在这里插入图片描述

🍑四、web页面一览

🍊登录

🍓登录地址：http://xxxx:8801/ui/#/login ; 用户密码 admin/admin
在这里插入图片描述
在这里插入图片描述

🍊数据源

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

🍊任务

在这里插入图片描述
选不到数据源…
在这里插入图片描述
在这里插入图片描述
🍓尝试创建整库同步
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

🍊用户管理

在这里插入图片描述

🍊虚拟表

在这里插入图片描述

🍑五、其它

🍊问题记录

🍓服务启动失败：Application run failed org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor

14:29:08.032 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor
 in 'reader', line 30, column 15:
        password: &R7#J*vDVWEuT48m
                  ^
unexpected character found *(42)
 in 'reader', line 30, column 20:
        password: &R7#J*vDVWEuT48m
                       ^

        at org.yaml.snakeyaml.scanner.ScannerImpl.scanAnchor(ScannerImpl.java:1507)
        at org.yaml.snakeyaml.scanner.ScannerImpl.fetchAnchor(ScannerImpl.java:958)
        at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:389)
        at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:248)
        at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:665)
        at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:165)
        at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:59)
        at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:45)
        at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:140)
        at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:119)
        at org.yaml.snakeyaml.composer.Composer.composeScalarN

🍓解决办法，密码&R7#J*vDVWEuT48m增加双引号
在这里插入图片描述
在这里插入图片描述

🍊总结

  • 🍓apache-seatunnel-web-1.0.0-bin还处在开发早期阶段，并不成熟，功能也不完善
  • 🍓安装seatunnel的目的，就是想参考下web界面，寻求针对表的拖拽处理（减少sql编写），目前看效果不理想，也只是一个简单的输入、输出表字段映射，因此暂告一段落
    在这里插入图片描述
韧小钊
关注
  • 25
    点赞
  • 踩
  • 38
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 8
    评论
专栏目录
Seatunnel源码解析(6) -Web接口启动Seatunnel
xd1753762376的博客
04-09 4857
Seatunnle(源waterdrop)，源码解析，二次封装
apache-hive-2.3.3-bin.tar.gz
04-02
`apache-hive-2.3.3-bin.tar.gz` 是Apache Hive 2.3.3版本的源码或二进制发行版的压缩包，通常包含了运行和管理Hive所需的所有文件。 在这个版本中，我们可能会发现以下关键组件和功能： 1. **Hive Metastore**：...
白鲸开源贡献 Apache SeaTunnel Web 界面
weixin_54625990的博客
05-17 3436
近日，白鲸开源在 Apache SeaTunnel 周会中宣布把 Web 界面贡献到社区，并制作了短视频简单介绍此次贡献将提供的功能，视频如下????：接下来我带大家了解一下有那些功能。首先演示一下：同步任务定义，可以在这里对需要的任务进行创建于管理。通过画布可以非常直观地观测到当前任务的状态，同时可以检查和修改参数配置，也可快速地查看表结构。同步任务实例，可以看到离线或实时的状态情况。更多关于 Sea...
部署SeaTunnel分布式集群
zhengzaifeidelushang的博客
05-02 5832
基于Hazelcast，SeaTunnel Engine 集群是运行 SeaTunnel Engine Server 的集群成员网络。只需要将$SEATUNNEL_HOMESeaTunnel Engine节点上的目录复制到Client节点，并配置SEATUNNEL_HOME类似的SeaTunnel Engine Server节点即可。这个参数主要是用来做集群隔离的，我们可以用这个来区分不同的集群，比如cluster1，cluster2，这个也用来区分不同的业务。N是集群节点的编号。
Hazelcast 分布式缓存 在Seatunnel中的使用
最新发布
TNT_D的博客
06-17 782
本文只是介绍了hazelcast的最基本用法，如果按照案例中的使用，完全可以用redis或者本地缓存。但是如果有了更高级（实际中的使用），那么hazelcast的分布式计算特性还是很好用的。源码也只是分析了本地的调用。如果感兴趣其实可以debug跟进去看下远程调用的方式。其实想想本质还是一样，远程调用就需要1、发现节点；2、注册节点；3、网络调用其他节点。而seatunnel的调用就相对来说更高级一些，它进行了一系列的封装。最后也还是网络调用其他节点。
一文搞定 Apache SeaTunnel 2.3.1 全流程部署使用
weixin_54625990的博客
06-07 2517
点击蓝字 关注我们作者 |caoyongfei部署1下载解压https://dlcdn.apache.org/incubator/SeaTunnel/2.3.1/apache-SeaTunnel-incubating-2.3.1-bin.tar.gz下载完毕之后上传到服务器上面并解压# 解压到了/opt/module目录下 tar -zxvf apache-SeaTunnel-incubatin...
CentOs7.x安装部署SeaTunnelWeb遇到的坑
qq_34905631的博客
12-19 3111
本次分享就到此结束了，踩了一些坑，还是请教了官方的大佬才知道的是哪里有问题，希望我的分享对你有所帮助，请一键三连，么么么哒！
Seatunnel单机和集群部署说明
u010479989的博客
11-14 891
将seatunnel-dist\target目录下的apache-seatunnel-2.3.3-SNAPSHOT-bin.tar.gz上传至服务器。$SEATUNNEL_HOME/bin/stop-seatunnel-cluster.sh -- 每个节点停止集群。./bin/seatunnel-cluster.sh -d -- 每个节点启动集群。解压安装包到/opt/soft/seatunnel目录下。
解读重要功能特性：新手入门 Apache SeaTunnel CDC
weixin_54625990的博客
01-04 4788
举个例子，我们前面讲了可以利用 CDC 将 MySQL 的数据同步到 Elasticsearch 中以供搜索，在这样的架构里，所有的查询都用 ES 来查，但在想修改数据时，并不直接修改 ES 里的数据，而是修改上游的 MySQL 数据，使之产生数据更新事件，事件被消费者消费来更新 ES 中的数据，这就基本上是一种 CQRS 模式。另外包括社区的展望和目前在做的一些事情。这个情况下，小数量的表手写还可以应付，如果数量大可能就出现类型映射错误的问题，或者参数配置错误的问题，就会产生很高的运维成本（配置麻烦）。
Seatunnel超高性能分布式数据集成平台使用体会
itxiaoshen博客
08-27 6814
有Sqoop和DataX之类数据处理为何还要用Apache SeaTunnel，这就要得益于Apache SeaTunnel依赖Flink和Spark天然分布式处理数据的特性，前两者是单机同步数据不适于海量数据同步，以低代码方式用配置文件就可以启动Flink数据处理应用，本篇从基本概念和原理入手，并通过部署SeaTunnel演示了多个基于Flink的Source和Sink配置，基本掌握如何编写配置使用。............
3 分钟部署 SeaTunnel Zeta 单节点 Standalone 模式环境
weixin_54625990的博客
05-16 1052
点击蓝字 关注我们部署环境（MacOS/Linux）1.安装JDK环境目前SeaTunnel支持运行在JDK8及以上的环境。用户需要自行安装JDK环境。2.下载安装包目前SeaTunnel的最新版本是2.3.1版本。这里我们安装部署2.3.1版本，如果你需要下载其它版本，可以从https://seatunnel.apache.org/download中查询对应的版本。mkdir ~/seatunn...
Seatunnel系列之：部署Seatunnel
zhengzaifeidelushang的博客
03-13 1983
编辑config/v2.batch.config.template，决定了seatunnel启动后数据输入、处理、输出的方式和逻辑。以下是配置文件的示例，与上面提到的示例应用程序相同。env {source {schema = {fields {age = agesink {Console {编辑config/v2.streaming.conf.template，决定了seatunnel启动后数据输入、处理、输出的方式和逻辑。以下是配置文件的示例，与上面提到的示例应用程序相同。env {
seatunnel-web-main
09-18
SeaTunnel-UI数据集成系统 系统主要有任务配置，任务实例，数据源，虚拟表，用户管理等功能
solr-8.6.3.tgz+hbase-2.3.3-bin.tar.gz
06-18
标题中的"solr-8.6.3.tgz+hbase-2.3.3-bin.tar.gz"表明我们有两个重要的开源软件版本：Apache Solr 8.6.3和HBase 2.3.3。Solr是Apache软件基金会的一个项目，主要用于全文搜索、企业级搜索和大数据分析。而HBase则是...
spring-boot-starter-web-2.3.3.RELEASE.jar
04-21
spring-boot-starter-web-1.0.0.RELEASE.jar 各个版本，免费下载 spring-boot-starter-web.RELEASE.jar 各个版本，免费下载 spring-boot-starter-web.jar 各个版本，免费下载 如果不能免费下载，关注我，评论区联系...
apache-atlas-2.1.0-hbase-hook.tar.gz
05-10
这个tar.gz压缩包是为在CDH6.3.1（Cloudera Data Hub 6.3.1）平台上构建和部署这个Hook而准备的。 HBase是一个分布式的、面向列的NoSQL数据库，它是Apache Hadoop生态系统的一部分，用于存储大规模结构化数据。...
nagios-plugins-2.3.3.tar.gz
11-05
"nagios-plugins-2.3.3.tar.gz" 是一个包含了Nagios官方插件的压缩包，版本为2.3.3，是经过广泛测试和验证的稳定版本。 在深入探讨Nagios插件之前，我们需要了解Nagios的基本架构。Nagios服务器会定期或按需向各个...
SeaTunnel安装及测试
dwjlyl
12-09 5598
SeaTunnel安装及测试
Apache SeaTunnel：新一代高性能、分布式、海量数据集成工具从入门到实践
分享云原生大数据技术和个人成长、职业发展相关内容
11-08 7768
本文介绍Apache SeaTunnel 以及关于Apache SeaTunnel的部署和使用。 Apache SeaTunnel是新一代高性能、分布式、海量数据集成工具，支持上百种数据源 ( Database/Cloud/SaaS ) 支持海量数据的实时 CDC 和批量同步，可以稳定高效地同步万亿级数据，
hyperledger-fabric-linux-amd64-2.3.3.tar.gz 下载
06-06
Hyperledger Fabric是一个开源的区块链框架，能够支持企业级应用程序的开发和部署。对于想要使用Hyperledger Fabric搭建自己的区块链应用的开发者来说，下载Hyperledger Fabric是第一步。 hyperledger-fabric-linux-amd64-2.3.3.tar.gz是一个用于Linux系统的Hyperledger Fabric 2.3.3版本的压缩文件，它包含了在Linux系统上使用Hyperledger Fabric所必需的所有文件和资源。 下载这个文件的方法很简单，只需在网上搜索下载链接即可。但在下载之前，需要确保所下载的文件是官方发布的版本，并且证书是经过验证的，以确保其安全性和完整性。 下载完之后，需要将文件解压缩并安装相应的组件和依赖项，以便在Linux系统上成功运行Hyperledger Fabric。运行程序之前，还需要做好一些设置和配置，包括定义身份和角色、创建通道和节点、安装链码等，这些都需要开发者具备一定的技能和经验。 总之，下载Hyperledger Fabric是开始使用这个区块链框架的关键步骤之一，但它只是整个过程的一部分，开发者还需要深入了解如何使用Hyperledger Fabric来构建他们的应用程序，这需要耐心和持续的学习。
评论 8
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

韧小钊

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值