Apache Seatunnel部署(apache-seatunnel-2.3.3-bin.tar.gz + apache-seatunnel-web-1.0.0-bin.tar.gz)

🍑一、概要


安装包及源码包下载地址
在这里插入图片描述
在这里插入图片描述

Apache SeaTunnel 是一个分布式、高性能、易扩展的数据集成平台,用于实时和离线数据处理。它支持多种数据源之间的数据迁移和转换。其中,apache-seatunnel-web-1.0.0-bin.tar.gzapache-seatunnel-2.3.3-bin.tar.gz 分别代表了 Apache SeaTunnel Web 界面的安装包和 Apache SeaTunnel 的核心引擎的安装包。

  • apache-seatunnel-web-1.0.0-bin.tar.gz:这是 Apache SeaTunnel Web 的安装包,提供了用户友好的 Web 界面,使得用户可以更方便地管理和监控 SeaTunnel 任务,包括创建、编辑和运行作业等操作。通过这个界面,用户可以无需直接接触命令行就能进行配置和管理。
  • apache-seatunnel-2.3.3-bin.tar.gz:这是 Apache SeaTunnel 核心引擎的二进制发行版,包含了执行数据抽取、转换和加载的核心代码库。它是实际执行数据处理任务的部分,可以与各种数据源和目标(如数据库、文件系统、消息队列等)进行交互。

在部署 Apache SeaTunnel 时,通常需要先安装并配置好核心引擎,然后根据需要选择是否使用 Web 界面进行管理。如果你只需要在命令行下运行和管理任务,那么只需要安装核心引擎即可。如果你想拥有一个图形化的任务管理界面,那么就需要同时安装 Apache SeaTunnel Web

🍑二、核心引擎部署


🍊 上传解压安装包: tar xf apache-seatunnel-2.3.3-bin.tar.gz
在这里插入图片描述

🍊 自动安装

🍓自动安装插件(全量安装,很慢,一直到下班,一个半小时都没结束,第二天早上来显示下载结束

[opensource@bigdata02 bin]$ pwd
/home/opensource/app/apache-seatunnel-2.3.3/bin
[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ ./install-plugin.sh 

在这里插入图片描述

🍓 验证服务

🍒执行系统自带的任务进行验证: sh seatunnel.sh --config ../config/v2.batch.config.template -m local
在这里插入图片描述

[opensource@bigdata02 bin]$ sh seatunnel.sh --config ../config/v2.batch.config.template -m local 
十二月 08, 2023 9:09:51 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:09:51 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:09:51 上午 org.apache.seatunnel.engine.common.config.SeaTunnelConfig
信息: seatunnel.home is /home/opensource/app/apache-seatunnel-2.3.3/bin
十二月 08, 2023 9:09:51 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:09:51 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
2023-12-08 09:09:51,483 WARN  com.hazelcast.instance.AddressPicker - [LOCAL] [seatunnel-233929] [5.1] You configured your member address as host name. Please be aware of that your dns can be spoofed. Make sure that your dns configurations are correct.
2023-12-08 09:09:51,487 INFO  com.hazelcast.instance.AddressPicker - [LOCAL] [seatunnel-233929] [5.1] Resolving domain name 'localhost' to address(es): [127.0.0.1]
2023-12-08 09:09:51,487 INFO  com.hazelcast.instance.AddressPicker - [LOCAL] [seatunnel-233929] [5.1] Interfaces is disabled, trying to pick one address from TCP-IP config addresses: [localhost/127.0.0.1]
2023-12-08 09:09:51,509 INFO  org.apache.seatunnel.engine.server.SeaTunnelServer - SeaTunnel server start...
2023-12-08 09:09:51,510 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] Based on Hazelcast IMDG version: 5.1.0 (20220228 - 21f20e7)
2023-12-08 09:09:51,510 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] Cluster name: seatunnel-233929
2023-12-08 09:09:51,510 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] 
                                                         
 _____               _____                             _ 
/  ___|             |_   _|                           | |
\ `--.   ___   __ _   | |   _   _  _ __   _ __    ___ | |
 `--. \ / _ \ / _` |  | |  | | | || '_ \ | '_ \  / _ \| |
/\__/ /|  __/| (_| |  | |  | |_| || | | || | | ||  __/| |
\____/  \___| \__,_|  \_/   \__,_||_| |_||_| |_| \___||_|
                                                         

2023-12-08 09:09:51,511 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] Copyright © 2021-2022 The Apache Software Foundation. Apache SeaTunnel, SeaTunnel, and its feather logo are trademarks of The Apache Software Foundation.
2023-12-08 09:09:51,511 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] Integrity Checker is disabled. Fail-fast on corrupted executables will not be performed.
To enable integrity checker do one of the following: 
  - Change member config using Java API: config.setIntegrityCheckerEnabled(true);
  - Change XML/YAML configuration property: Set hazelcast.integrity-checker.enabled to true
  - Add system property: -Dhz.integritychecker.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_INTEGRITYCHECKER_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2023-12-08 09:09:51,513 INFO  com.hazelcast.system - [localhost]:5801 [seatunnel-233929] [5.1] The Jet engine is disabled.
To enable the Jet engine on the members, do one of the following:
  - Change member config using Java API: config.getJetConfig().setEnabled(true)
  - Change XML/YAML configuration property: Set hazelcast.jet.enabled to true
  - Add system property: -Dhz.jet.enabled=true (for Hazelcast embedded, works only when loading config via Config.load)
  - Add environment variable: HZ_JET_ENABLED=true (recommended when running container image. For Hazelcast embedded, works only when loading config via Config.load)
2023-12-08 09:09:51,867 INFO  com.hazelcast.system.security - [localhost]:5801 [seatunnel-233929] [5.1] Enable DEBUG/FINE log level for log category com.hazelcast.system.security  or use -Dhazelcast.security.recommendations system property to see  security recommendations and the status of current config.
2023-12-08 09:09:51,919 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-233929] [5.1] Using TCP/IP discovery
2023-12-08 09:09:51,920 WARN  com.hazelcast.cp.CPSubsystem - [localhost]:5801 [seatunnel-233929] [5.1] CP Subsystem is not enabled. CP data structures will operate in UNSAFE mode! Please note that UNSAFE mode will not provide strong consistency guarantees.
十二月 08, 2023 9:09:52 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:09:52 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:09:52 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:09:52 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
2023-12-08 09:09:52,071 WARN  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] The Node is not ready yet, Node state STARTING,looking forward to the next scheduling
2023-12-08 09:09:52,072 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] Created new BusWork : 1217639236
2023-12-08 09:09:52,082 WARN  org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - failed send heartbeat to resource manager, will retry later. this address: [localhost]:5801
2023-12-08 09:09:52,084 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2023-12-08 09:09:52,121 INFO  com.hazelcast.internal.diagnostics.Diagnostics - [localhost]:5801 [seatunnel-233929] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2023-12-08 09:09:52,128 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5801 is STARTING
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5802 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5817 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5819 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5823 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5804 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5821 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5813 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5825 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5815 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5808 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5806 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5829 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5810 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5827 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5822 is added to the blacklist.
2023-12-08 09:09:52,166 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5820 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5814 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5812 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5826 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5805 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5807 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5818 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5816 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5830 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5809 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5811 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5824 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5828 is added to the blacklist.
2023-12-08 09:09:52,167 INFO  com.hazelcast.internal.cluster.impl.TcpIpJoiner - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5803 is added to the blacklist.
2023-12-08 09:09:53,162 INFO  com.hazelcast.internal.cluster.ClusterService - [localhost]:5801 [seatunnel-233929] [5.1] 

Members {size:1, ver:1} [
        Member [localhost]:5801 - 87c80e4a-80c5-4b82-b577-cb0649025b5e this
]

2023-12-08 09:09:53,172 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5801 is STARTED
十二月 08, 2023 9:09:53 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast-client.yaml' from System property 'hazelcast.client.config'
十二月 08, 2023 9:09:53 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast-client.yaml
2023-12-08 09:09:53,269 INFO  com.hazelcast.client.impl.spi.ClientInvocationService - hz.client_1 [seatunnel-233929] [5.1] Running with 2 response threads, dynamic=true
2023-12-08 09:09:53,279 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTING
2023-12-08 09:09:53,279 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is STARTED
2023-12-08 09:09:53,285 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-233929] [5.1] Trying to connect to cluster: seatunnel-233929
2023-12-08 09:09:53,286 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-233929] [5.1] Trying to connect to [localhost]:5801
2023-12-08 09:09:53,299 INFO  com.hazelcast.client.impl.protocol.task.AuthenticationMessageTask - [localhost]:5801 [seatunnel-233929] [5.1] Received auth from Connection[id=1, /127.0.0.1:5801->/127.0.0.1:55059, qualifier=null, endpoint=[127.0.0.1]:55059, remoteUuid=535a4a1b-58b7-40a4-8909-218d69fedfe5, alive=true, connectionType=JVM, planeIndex=-1], successfully authenticated, clientUuid: 535a4a1b-58b7-40a4-8909-218d69fedfe5, client name: hz.client_1, client version: 5.1
2023-12-08 09:09:53,302 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_CONNECTED
2023-12-08 09:09:53,302 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-233929] [5.1] Authenticated with server [localhost]:5801:87c80e4a-80c5-4b82-b577-cb0649025b5e, server version: 5.1, local address: /127.0.0.1:55059
2023-12-08 09:09:53,303 INFO  com.hazelcast.internal.diagnostics.Diagnostics - hz.client_1 [seatunnel-233929] [5.1] Diagnostics disabled. To enable add -Dhazelcast.diagnostics.enabled=true to the JVM arguments.
2023-12-08 09:09:53,308 INFO  com.hazelcast.client.impl.spi.ClientClusterService - hz.client_1 [seatunnel-233929] [5.1] 

Members [1] {
        Member [localhost]:5801 - 87c80e4a-80c5-4b82-b577-cb0649025b5e
}

2023-12-08 09:09:53,329 INFO  com.hazelcast.client.impl.statistics.ClientStatisticsService - Client statistics is enabled with period 5 seconds.
2023-12-08 09:09:53,453 INFO  org.apache.seatunnel.engine.client.job.JobExecutionEnvironment - add common jar in plugins :[]
2023-12-08 09:09:53,466 INFO  org.apache.seatunnel.core.starter.utils.ConfigBuilder - Loading config file from path: ../config/v2.batch.config.template
2023-12-08 09:09:53,504 INFO  org.apache.seatunnel.core.starter.utils.ConfigShadeUtils - Load config shade spi: [base64]
2023-12-08 09:09:53,540 INFO  org.apache.seatunnel.core.starter.utils.ConfigBuilder - Parsed config file: {
    "env" : {
        "execution.parallelism" : 2,
        "job.mode" : "BATCH",
        "checkpoint.interval" : 10000
    },
    "source" : [
        {
            "schema" : {
                "fields" : {
                    "name" : "string",
                    "age" : "int"
                }
            },
            "row.num" : 16,
            "parallelism" : 2,
            "result_table_name" : "fake",
            "plugin_name" : "FakeSource"
        }
    ],
    "sink" : [
        {
            "plugin_name" : "Console"
        }
    ]
}

2023-12-08 09:09:53,560 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,561 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,564 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSink Plugin from /home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel
2023-12-08 09:09:53,568 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: FakeSource at: file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-fake-2.3.3.jar
2023-12-08 09:09:53,568 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: Console at: file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-console-2.3.3.jar
2023-12-08 09:09:53,571 INFO  org.apache.seatunnel.engine.core.parse.ConfigParserUtil - Currently, incorrect configuration of source_table_name and result_table_name options don't affect job running. In the future we will ban incorrect configurations.
2023-12-08 09:09:53,571 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,571 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,571 WARN  org.apache.seatunnel.engine.core.parse.ConfigParserUtil - This configuration is not recommended. A source/transform(FakeSource) is configured with 'result_table_name' option value of 'fake', but subsequent transform/sink(Console) is not configured with 'source_table_name' option.
2023-12-08 09:09:53,572 INFO  org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser - start generating all sources.
2023-12-08 09:09:53,572 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,581 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load SeaTunnelSource Plugin from /home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel
2023-12-08 09:09:53,585 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Discovery plugin jar: FakeSource at: file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-fake-2.3.3.jar
2023-12-08 09:09:53,586 INFO  org.apache.seatunnel.plugin.discovery.AbstractPluginDiscovery - Load plugin: PluginIdentifier{engineType='seatunnel', pluginType='source', pluginName='FakeSource'} from classpath
2023-12-08 09:09:53,603 INFO  org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser - start generating all transforms.
2023-12-08 09:09:53,603 INFO  org.apache.seatunnel.engine.core.parse.MultipleTableJobConfigParser - start generating all sinks.
2023-12-08 09:09:53,603 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,607 INFO  org.apache.seatunnel.api.configuration.ReadonlyConfig - Config uses fallback configuration key 'plugin_name' instead of key 'factory'
2023-12-08 09:09:53,636 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Start submit job, job id: 785312093409640449, with plugin jar [file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-fake-2.3.3.jar, file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-console-2.3.3.jar]
2023-12-08 09:09:53,640 WARN  org.apache.seatunnel.engine.server.SeaTunnelServer - This is master node, waiting the coordinator service init finished
2023-12-08 09:09:54,086 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:09:54,114 INFO  com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel-233929] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:09:54,141 WARN  org.apache.seatunnel.engine.server.SeaTunnelServer - This is master node, waiting the coordinator service init finished
2023-12-08 09:09:54,646 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:09:54,646 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker... 
2023-12-08 09:09:54,646 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:09:54,648 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=1029177344}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=1029177344}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:09:54,662 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Init JobMaster for Job SeaTunnel_Job (785312093409640449) 
2023-12-08 09:09:54,662 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job SeaTunnel_Job (785312093409640449) needed jar urls [file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-fake-2.3.3.jar, file:/home/opensource/app/apache-seatunnel-2.3.3/connectors/seatunnel/connector-console-2.3.3.jar]
2023-12-08 09:09:54,874 WARN  org.apache.hadoop.util.NativeCodeLoader - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
2023-12-08 09:09:54,976 INFO  org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage - Path /tmp/seatunnel/checkpoint_snapshot/785312093409640449 is not a directory
2023-12-08 09:09:54,976 INFO  org.apache.seatunnel.engine.checkpoint.storage.hdfs.HdfsStorage - No checkpoint found for job, job id is: 785312093409640449
2023-12-08 09:09:54,978 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] is in state CREATED when init state future
2023-12-08 09:09:54,979 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] is in state CREATED when init state future
2023-12-08 09:09:54,979 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - The task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] is in state CREATED when init state future
2023-12-08 09:09:54,981 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Submit job finished, job id: 785312093409640449, job name: SeaTunnel
2023-12-08 09:09:54,982 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job Job SeaTunnel_Job (785312093409640449) waiting for scheduler finished
2023-12-08 09:09:54,984 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (785312093409640449) turn from state CREATED to SCHEDULED.
2023-12-08 09:09:54,987 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] turn from state CREATED to SCHEDULED.
2023-12-08 09:09:54,989 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] state from CREATED to SCHEDULED
2023-12-08 09:09:54,990 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] turn from state CREATED to SCHEDULED.
2023-12-08 09:09:54,994 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] state from CREATED to SCHEDULED
2023-12-08 09:09:54,994 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] turn from state CREATED to SCHEDULED.
2023-12-08 09:09:54,995 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] state from CREATED to SCHEDULED
2023-12-08 09:09:54,996 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] turn from state CREATED to SCHEDULED.
2023-12-08 09:09:55,003 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] turn from state SCHEDULED to DEPLOYING.
2023-12-08 09:09:55,005 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] state from SCHEDULED to DEPLOYING
2023-12-08 09:09:55,006 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] turn from state SCHEDULED to DEPLOYING.
2023-12-08 09:09:55,006 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] state from SCHEDULED to DEPLOYING
2023-12-08 09:09:55,007 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] turn from state SCHEDULED to DEPLOYING.
2023-12-08 09:09:55,008 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] state from SCHEDULED to DEPLOYING
2023-12-08 09:09:55,009 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] turn from state SCHEDULED to DEPLOYING.
2023-12-08 09:09:55,009 WARN  org.apache.seatunnel.engine.client.job.JobMetricsRunner - Failed to get job metrics summary, it maybe first-run
2023-12-08 09:09:55,016 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] received deploying task executionId [785312099889840129]
2023-12-08 09:09:55,025 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] received deploying task executionId [785312099889971201]
2023-12-08 09:09:55,025 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] received deploying task executionId [785312099889905665]
2023-12-08 09:09:55,034 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1}, executionId [785312099889840129]
2023-12-08 09:09:55,035 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1} init success
2023-12-08 09:09:55,040 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1} success
2023-12-08 09:09:55,040 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] state from DEPLOYING to RUNNING
2023-12-08 09:09:55,042 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - starting seatunnel source split enumerator task, source name: pipeline-1 [Source[0]-FakeSource-fake]
2023-12-08 09:09:55,042 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] turn from state DEPLOYING to RUNNING.
2023-12-08 09:09:55,047 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}, executionId [785312099889971201]
2023-12-08 09:09:55,048 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001} init success
2023-12-08 09:09:55,048 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}, executionId [785312099889905665]
2023-12-08 09:09:55,049 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001} success
2023-12-08 09:09:55,050 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] state from DEPLOYING to RUNNING
2023-12-08 09:09:55,051 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000} init success
2023-12-08 09:09:55,052 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] turn from state DEPLOYING to RUNNING.
2023-12-08 09:09:55,052 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] deploying TaskGroup TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000} success
2023-12-08 09:09:55,053 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Try to update the task Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] state from DEPLOYING to RUNNING
2023-12-08 09:09:55,054 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] turn from state DEPLOYING to RUNNING.
2023-12-08 09:09:55,054 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] turn from state DEPLOYING to RUNNING.
2023-12-08 09:09:55,054 INFO  org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask - starting seatunnel transform task, index 0
2023-12-08 09:09:55,054 INFO  org.apache.seatunnel.engine.server.task.TransformSeaTunnelTask - starting seatunnel transform task, index 1
2023-12-08 09:09:55,060 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (785312093409640449) turn from state SCHEDULED to RUNNING.
2023-12-08 09:09:55,061 INFO  org.apache.seatunnel.engine.server.master.JobMaster - Job SeaTunnel_Job (785312093409640449) scheduler finished
2023-12-08 09:09:55,064 INFO  org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask - starting seatunnel source task, index 0
2023-12-08 09:09:55,064 INFO  org.apache.seatunnel.engine.server.task.SourceSeaTunnelTask - starting seatunnel source task, index 1
2023-12-08 09:09:55,068 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - output rowType: name<STRING>, age<INT>
2023-12-08 09:09:55,068 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - output rowType: name<STRING>, age<INT>
2023-12-08 09:09:55,166 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}, taskID=40001, index=1}
2023-12-08 09:09:55,166 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received reader register, readerID: TaskLocation{taskGroupLocation=TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}, taskID=40000, index=0}
2023-12-08 09:09:55,245 INFO  org.apache.seatunnel.engine.server.task.SourceSplitEnumeratorTask - received enough reader, starting enumerator...
2023-12-08 09:09:55,245 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Starting to calculate splits.
2023-12-08 09:09:55,246 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Assigned [FakeSourceSplit(splitId=1, rowNum=16), FakeSourceSplit(splitId=0, rowNum=16)] to 2 readers.
2023-12-08 09:09:55,246 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Calculated splits successfully, the size of splits is 2.
2023-12-08 09:09:55,247 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Assigning splits to readers 0 [FakeSourceSplit(splitId=0, rowNum=16)]
2023-12-08 09:09:55,251 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceSplitEnumerator - Assigning splits to readers 1 [FakeSourceSplit(splitId=1, rowNum=16)]
2023-12-08 09:09:55,269 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceReader - 16 rows of data have been generated in split(0). Generation time: 1701997795267
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceReader - Closed the bounded fake source
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceReader - 16 rows of data have been generated in split(1). Generation time: 1701997795266
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.fake.source.FakeSourceReader - Closed the bounded fake source
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=1:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : kkTzr, 348993738
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=1:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : cHRlJ, 141341247
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=2:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : JoqKS, 949667288
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=2:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : wuRPB, 739004063
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=3:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : vzyWZ, 1185127730
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=3:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : MzsIj, 255990586
2023-12-08 09:09:55,270 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=4:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : pKInq, 1307779433
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=4:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : JmaUB, 255860364
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=5:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : MYeRA, 1997823969
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=5:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : uVILC, 995215612
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=6:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : DlVbk, 731828921
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=7:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : inYyz, 1847828742
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=6:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : cyjxD, 1832606505
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=7:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : eMzXN, 802135106
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=8:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : kPUMq, 1916092538
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=8:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : aySgR, 1299495336
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=9:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : vFyQU, 1895773363
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=9:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : pjYqn, 1385074915
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=10:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : dQmgk, 1234357286
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=10:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : zASIv, 1497863961
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=11:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : zQuTA, 903385081
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=11:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : IVXOt, 870847193
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=12:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : jrLTl, 705955925
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=13:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : YflRj, 293817756
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=12:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : DbvZU, 1486490099
2023-12-08 09:09:55,271 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=14:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : SeaoD, 1066109912
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=13:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : VsbMy, 910565833
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=15:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : vkEhB, 403795863
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=14:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : fegPV, 165600914
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=0  rowIndex=16:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : rXrEp, 1087983282
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=15:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : DokOA, 1281308771
2023-12-08 09:09:55,272 INFO  org.apache.seatunnel.connectors.seatunnel.console.sink.ConsoleSinkWriter - subtaskIndex=1  rowIndex=16:  SeaTunnelRow#tableId= SeaTunnelRow#kind=INSERT : VjDeD, 1091404164
2023-12-08 09:09:55,356 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - wait checkpoint completed: 9223372036854775807
2023-12-08 09:09:55,411 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - pending checkpoint(9223372036854775807/1@785312093409640449) notify finished!
2023-12-08 09:09:55,411 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start notify checkpoint completed, checkpoint:org.apache.seatunnel.engine.server.checkpoint.CompletedCheckpoint@4d1c18a2
2023-12-08 09:09:55,416 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - start clean pending checkpoint cause CheckpointCoordinator completed.
2023-12-08 09:09:55,417 INFO  org.apache.seatunnel.engine.server.checkpoint.CheckpointCoordinator - Turn checkpoint_state_785312093409640449_1 state from null to FINISHED
2023-12-08 09:09:55,453 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 20000, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1}
2023-12-08 09:09:55,453 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] Task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1} complete with state FINISHED
2023-12-08 09:09:55,454 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] Received task end from execution TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=1}, state FINISHED
2023-12-08 09:09:55,455 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] turn to end state FINISHED.
2023-12-08 09:09:55,455 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SplitEnumerator (1/1)] end with state FINISHED
2023-12-08 09:09:55,464 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 50001, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}
2023-12-08 09:09:55,464 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 50000, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}
2023-12-08 09:09:56,271 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 40001, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}
2023-12-08 09:09:56,271 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] taskDone, taskId = 40000, taskGroup = TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}
2023-12-08 09:09:56,272 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] Task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000} complete with state FINISHED
2023-12-08 09:09:56,272 INFO  org.apache.seatunnel.engine.server.TaskExecutionService - [localhost]:5801 [seatunnel-233929] [5.1] Task TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001} complete with state FINISHED
2023-12-08 09:09:56,272 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] Received task end from execution TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30000}, state FINISHED
2023-12-08 09:09:56,272 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel-233929] [5.1] Received task end from execution TaskGroupLocation{jobId=785312093409640449, pipelineId=1, taskGroupId=30001}, state FINISHED
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] turn to end state FINISHED.
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] turn to end state FINISHED.
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (1/2)] end with state FINISHED
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalVertex - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)], task: [pipeline-1 [Source[0]-FakeSource-fake]-SourceTask (2/2)] end with state FINISHED
2023-12-08 09:09:56,273 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] end with state FINISHED
2023-12-08 09:09:56,310 INFO  org.apache.seatunnel.engine.server.master.JobMaster - release the pipeline Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] resource
2023-12-08 09:09:56,311 INFO  org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 785312093409640449, slot: SlotProfile{worker=[localhost]:5801, slotID=1, ownerJobID=785312093409640449, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='fa7bad5a-cd2c-4d97-8368-88027b3b49b6'}
2023-12-08 09:09:56,311 INFO  org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 785312093409640449, slot: SlotProfile{worker=[localhost]:5801, slotID=2, ownerJobID=785312093409640449, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='fa7bad5a-cd2c-4d97-8368-88027b3b49b6'}
2023-12-08 09:09:56,312 INFO  org.apache.seatunnel.engine.server.service.slot.DefaultSlotService - received slot release request, jobID: 785312093409640449, slot: SlotProfile{worker=[localhost]:5801, slotID=3, ownerJobID=785312093409640449, assigned=true, resourceProfile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=0}}, sequence='fa7bad5a-cd2c-4d97-8368-88027b3b49b6'}
2023-12-08 09:09:56,314 INFO  org.apache.seatunnel.engine.server.dag.physical.SubPlan - Job SeaTunnel_Job (785312093409640449), Pipeline: [(1/1)] turn to end state FINISHED.
2023-12-08 09:09:56,315 INFO  org.apache.seatunnel.engine.server.dag.physical.PhysicalPlan - Job SeaTunnel_Job (785312093409640449) end with state FINISHED
2023-12-08 09:09:56,328 INFO  org.apache.seatunnel.engine.client.job.ClientJobProxy - Job (785312093409640449) end with state FINISHED
2023-12-08 09:09:56,353 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - 
***********************************************
           Job Statistic Information
***********************************************
Start Time                : 2023-12-08 09:09:53
End Time                  : 2023-12-08 09:09:56
Total Time(s)             :                   2
Total Read Count          :                  32
Total Write Count         :                  32
Total Failed Count        :                   0
***********************************************

2023-12-08 09:09:56,354 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTTING_DOWN
2023-12-08 09:09:56,357 INFO  com.hazelcast.internal.server.tcp.TcpServerConnection - [localhost]:5801 [seatunnel-233929] [5.1] Connection[id=1, /127.0.0.1:5801->/127.0.0.1:55059, qualifier=null, endpoint=[127.0.0.1]:55059, remoteUuid=535a4a1b-58b7-40a4-8909-218d69fedfe5, alive=false, connectionType=JVM, planeIndex=-1] closed. Reason: Connection closed by the other side
2023-12-08 09:09:56,357 INFO  com.hazelcast.client.impl.connection.ClientConnectionManager - hz.client_1 [seatunnel-233929] [5.1] Removed connection to endpoint: [localhost]:5801:87c80e4a-80c5-4b82-b577-cb0649025b5e, connection: ClientConnection{alive=false, connectionId=1, channel=NioChannel{/127.0.0.1:55059->localhost/127.0.0.1:5801}, remoteAddress=[localhost]:5801, lastReadTime=2023-12-08 09:09:56.351, lastWriteTime=2023-12-08 09:09:56.328, closedTime=2023-12-08 09:09:56.355, connected server version=5.1}
2023-12-08 09:09:56,358 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is CLIENT_DISCONNECTED
2023-12-08 09:09:56,358 INFO  com.hazelcast.client.impl.ClientEndpointManager - [localhost]:5801 [seatunnel-233929] [5.1] Destroying ClientEndpoint{connection=Connection[id=1, /127.0.0.1:5801->/127.0.0.1:55059, qualifier=null, endpoint=[127.0.0.1]:55059, remoteUuid=535a4a1b-58b7-40a4-8909-218d69fedfe5, alive=false, connectionType=JVM, planeIndex=-1], clientUuid=535a4a1b-58b7-40a4-8909-218d69fedfe5, clientName=hz.client_1, authenticated=true, clientVersion=5.1, creationTime=1701997793296, latest clientAttributes=lastStatisticsCollectionTime=1701997793329,enterprise=false,clientType=JVM,clientVersion=5.1,clusterConnectionTimestamp=1701997793287,clientAddress=127.0.0.1,clientName=hz.client_1,credentials.principal=null,os.committedVirtualMemorySize=24435175424,os.freePhysicalMemorySize=50645405696,os.freeSwapSpaceSize=0,os.maxFileDescriptorCount=65536,os.openFileDescriptorCount=58,os.processCpuTime=9670000000,os.systemLoadAverage=2.31,os.totalPhysicalMemorySize=540422402048,os.totalSwapSpaceSize=0,runtime.availableProcessors=104,runtime.freeMemory=959249984,runtime.maxMemory=1029177344,runtime.totalMemory=1029177344,runtime.uptime=2770,runtime.usedMemory=69927360, labels=[]}
2023-12-08 09:09:56,360 INFO  com.hazelcast.core.LifecycleService - hz.client_1 [seatunnel-233929] [5.1] HazelcastClient 5.1 (20220228 - 21f20e7) is SHUTDOWN
2023-12-08 09:09:56,360 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed SeaTunnel client......
2023-12-08 09:09:56,360 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5801 is SHUTTING_DOWN
2023-12-08 09:09:56,363 INFO  com.hazelcast.internal.partition.impl.MigrationManager - [localhost]:5801 [seatunnel-233929] [5.1] Shutdown request of Member [localhost]:5801 - 87c80e4a-80c5-4b82-b577-cb0649025b5e this is handled
2023-12-08 09:09:56,367 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-233929] [5.1] Shutting down connection manager...
2023-12-08 09:09:56,369 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-233929] [5.1] Shutting down node engine...
2023-12-08 09:09:58,473 INFO  com.hazelcast.instance.impl.NodeExtension - [localhost]:5801 [seatunnel-233929] [5.1] Destroying node NodeExtension.
2023-12-08 09:09:58,474 INFO  com.hazelcast.instance.impl.Node - [localhost]:5801 [seatunnel-233929] [5.1] Hazelcast Shutdown is completed in 2111 ms.
2023-12-08 09:09:58,474 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel-233929] [5.1] [localhost]:5801 is SHUTDOWN
2023-12-08 09:09:58,474 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed HazelcastInstance ......
2023-12-08 09:09:58,474 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - Closed metrics executor service ......
2023-12-08 09:09:58,475 INFO  org.apache.seatunnel.core.starter.seatunnel.command.ClientExecuteCommand - run shutdown hook because get close signal
[opensource@bigdata02 bin]$

🍓 启动服务

🍒启动服务并且后台运行,用于web端连接:nohup sh seatunnel-cluster.sh 2>&1 &
在这里插入图片描述


[opensource@bigdata02 bin]$ ls
install-plugin.sh     seatunnel.sh                              start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh   stop-seatunnel-cluster.sh
[opensource@bigdata02 bin]$ 
[opensource@bigdata02 bin]$ nohup sh seatunnel-cluster.sh 2>&1 &    
[1] 93523
[opensource@bigdata02 bin]$ nohup: 忽略输入并把输出追加到"nohup.out"

[opensource@bigdata02 bin]$ ls
install-plugin.sh  seatunnel-cluster.sh  start-seatunnel-flink-13-connector-v2.sh  start-seatunnel-spark-2-connector-v2.sh  stop-seatunnel-cluster.sh
nohup.out          seatunnel.sh          start-seatunnel-flink-15-connector-v2.sh  start-seatunnel-spark-3-connector-v2.sh
[opensource@bigdata02 bin]$ tail -f nohup.out 
十二月 08, 2023 9:30:58 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml' from System property 'seatunnel.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/seatunnel.yaml
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Loading configuration '/home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml' from System property 'hazelcast.config'
十二月 08, 2023 9:30:59 上午 com.hazelcast.internal.config.AbstractConfigLocator
信息: Using configuration file at /home/opensource/app/apache-seatunnel-2.3.3/config/hazelcast.yaml
^C
[opensource@bigdata02 bin]$ tail -f ../logs/seatunnel-engine-server.log 
        Member [localhost]:5801 - 9db58520-2335-441c-8999-ad78123f6509 this
]

2023-12-08 09:30:59,930 INFO  com.hazelcast.core.LifecycleService - [localhost]:5801 [seatunnel] [5.1] [localhost]:5801 is STARTED
2023-12-08 09:31:01,865 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] This node become a new active master node, begin init coordinator service
2023-12-08 09:31:02,061 INFO  com.hazelcast.internal.partition.impl.PartitionStateManager - [localhost]:5801 [seatunnel] [5.1] Initializing cluster partition table arrangement...
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - Init ResourceManager
2023-12-08 09:31:04,863 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker... 
2023-12-08 09:31:04,864 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - initWorker live nodes: [[localhost]:5801]
2023-12-08 09:31:04,865 INFO  org.apache.seatunnel.engine.server.resourcemanager.AbstractResourceManager - registerWorker: {[localhost]:5801=WorkerProfile(address=[localhost]:5801, profile=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, unassignedResource=ResourceProfile{cpu=CPU{core=0}, heapMemory=Memory{bytes=2058354688}}, assignedSlots=[], unassignedSlots=[])}
2023-12-08 09:31:59,863 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
     CoordinatorService Thread Pool Status
***********************************************
activeCount               :                   0
corePoolSize              :                   0
maximumPoolSize           :          2147483647
poolSize                  :                   0
completedTaskCount        :                   0
taskCount                 :                   0
***********************************************

2023-12-08 09:31:59,867 INFO  org.apache.seatunnel.engine.server.CoordinatorService - [localhost]:5801 [seatunnel] [5.1] 
***********************************************
                Job info detail
***********************************************
createdJobCount           :                   0
scheduledJobCount         :                   0
runningJobCount           :                   0
failingJobCount           :                   0
failedJobCount            :                   0
cancellingJobCount        :                   0
canceledJobCount          :                   0
finishedJobCount          :                   0
restartingJobCount        :                   0
suspendedJobCount         :                   0
reconcilingJobCount       :                   0
***********************************************

🍊 手动安装

  • 不想全部安装,修改配置vi config/plugin_config具体保留哪些就不清楚了,只能说官方提供的安装包不好,无法直接启动,必须要先安装插件,但是这么多插件下载又很慢、也没必要
    在这里插入图片描述
  • 无法在线安装,只能手动了
    maven库下载地址
    在这里插入图片描述
    对于自动下载下来的jar包,已经全部打包上传到CSDN,大家可以免费下载,直接放到connectors/seatunnel
    在这里插入图片描述
    下载地址
    在这里插入图片描述
    CSDN资源大小不允许超过1000M,这里采用.tar.xz格式(解压会有些慢,小十分钟吧)
    • 三种压缩命令:
      tar -cf seatunnel_connectors_2.3.3.tar seatunnel
      tar -czf seatunnel_connectors_2.3.3.tar.gz seatunnel
      tar -cJf seatunnel_connectors_2.3.3.tar.xz seatunnel 
      
    • 对应解压命令:
      tar -xf seatunnel_connectors_2.3.3.tar 
      tar -xzf seatunnel_connectors_2.3.3.tar.gz 
      tar -xJf seatunnel_connectors_2.3.3.tar.xz 
      
      在这里插入图片描述

🍑三、Web界面部署


🍊 上传解压web部署包:tar xf apache-seatunnel-web-1.0.0-bin.tar.gz
在这里插入图片描述

🍊 初始化数据库

🍓 初始化SQL脚本script/seatunnel_server_mysql.sql
在这里插入图片描述
在这里插入图片描述

SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for role
-- ----------------------------
DROP TABLE IF EXISTS `role`;
CREATE TABLE `role`  (
  `id` int(20) NOT NULL AUTO_INCREMENT,
  `type` int(2) NOT NULL,
  `role_name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NULL DEFAULT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Records of role
-- ----------------------------
INSERT INTO `role`(`type`,`role_name`,`description`) values (0, 'ADMIN_ROLE', 'Admin User');
INSERT INTO `role`(`type`,`role_name`,`description`) values (1, 'NORMAL_ROLE', 'Normal User');

-- ----------------------------
-- Table structure for role_user_relation
-- ----------------------------
DROP TABLE IF EXISTS `role_user_relation`;
CREATE TABLE `role_user_relation`  (
  `id` int(20) NOT NULL AUTO_INCREMENT,
  `role_id` int(20) NOT NULL,
  `user_id` int(20) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_datasource
-- ----------------------------
DROP TABLE IF EXISTS `t_st_datasource`;
CREATE TABLE `t_st_datasource`  (
  `id` bigint(20) NOT NULL,
  `datasource_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `plugin_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `plugin_version` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT '1.0.0',
  `datasource_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `t_st_datasource_datasource_name_uindex`(`datasource_name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_definition
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_definition`;
CREATE TABLE `t_st_job_definition`  (
  `id` bigint(20) NOT NULL,
  `name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  UNIQUE INDEX `name`(`name`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_instance
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance`;
CREATE TABLE `t_st_job_instance`  (
  `id` bigint(20) NOT NULL,
  `job_define_id` bigint(20) NOT NULL,
  `job_status` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `job_config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `job_engine_id` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(20) NOT NULL,
  `update_user_id` int(20) NULL DEFAULT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  `end_time` timestamp(3) NULL DEFAULT NULL,
  `job_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_instance_history
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_instance_history`;
CREATE TABLE `t_st_job_instance_history`  (
  `id` bigint(20) NOT NULL,
  `dag` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_line
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_line`;
CREATE TABLE `t_st_job_line`  (
  `id` bigint(20) NOT NULL,
  `version_id` bigint(20) NOT NULL,
  `input_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `target_plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `job_line_version_index`(`version_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_metrics
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_metrics`;
CREATE TABLE `t_st_job_metrics`  (
  `id` bigint(20) NOT NULL,
  `job_instance_id` bigint(20) NOT NULL,
  `pipeline_id` int(20) NOT NULL,
  `read_row_count` bigint(20) NOT NULL,
  `write_row_count` bigint(20) NOT NULL,
  `source_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `sink_table_names` varchar(200) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `read_qps` bigint(20) NULL DEFAULT NULL,
  `write_qps` bigint(20) NULL DEFAULT NULL,
  `record_delay` bigint(20) NULL DEFAULT NULL,
  `status` varchar(20) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(20) NOT NULL,
  `update_user_id` int(20) NULL DEFAULT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_task
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_task`;
CREATE TABLE `t_st_job_task`  (
  `id` bigint(20) NOT NULL,
  `version_id` bigint(20) NOT NULL,
  `plugin_id` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `config` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `transform_options` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `output_schema` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `connector_type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `datasource_id` bigint(20) NULL DEFAULT NULL,
  `datasource_option` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `select_table_fields` varchar(5000) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `scene_mode` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `type` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE,
  INDEX `job_task_plugin_id_index`(`plugin_id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_job_version
-- ----------------------------
DROP TABLE IF EXISTS `t_st_job_version`;
CREATE TABLE `t_st_job_version`  (
  `id` bigint(20) NOT NULL,
  `job_id` bigint(20) NOT NULL,
  `name` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `job_mode` varchar(10) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `env` text CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL,
  `engine_name` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `engine_version` varchar(50) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for t_st_virtual_table
-- ----------------------------
DROP TABLE IF EXISTS `t_st_virtual_table`;
CREATE TABLE `t_st_virtual_table`  (
  `id` bigint(20) NOT NULL,
  `datasource_id` bigint(20) NOT NULL,
  `virtual_database_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `virtual_table_name` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `table_fields` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `virtual_table_config` varchar(1023) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NOT NULL,
  `description` varchar(63) CHARACTER SET utf8mb4 COLLATE utf8mb4_bin NULL DEFAULT NULL,
  `create_user_id` int(11) NOT NULL,
  `update_user_id` int(11) NOT NULL,
  `create_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` timestamp(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB CHARACTER SET = utf8mb4 COLLATE = utf8mb4_bin ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for user
-- ----------------------------
DROP TABLE IF EXISTS `user`;
CREATE TABLE `user`  (
  `id` int(11) NOT NULL AUTO_INCREMENT,
  `username` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `password` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `status` tinyint(4) NOT NULL,
  `type` tinyint(4) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 2 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Table structure for user_login_log
-- ----------------------------
DROP TABLE IF EXISTS `user_login_log`;
CREATE TABLE `user_login_log`  (
  `id` bigint(20) NOT NULL AUTO_INCREMENT,
  `user_id` int(11) NOT NULL,
  `token` mediumtext CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci NOT NULL,
  `token_status` tinyint(1) NOT NULL,
  `create_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3),
  `update_time` datetime(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3) ON UPDATE CURRENT_TIMESTAMP(3),
  PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 106 CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci ROW_FORMAT = Dynamic;

-- ----------------------------
-- Records of user_login_log
-- ----------------------------

INSERT INTO `user`(`username`,`password`,`status`,`type`) values ('admin', '7f97da8846fed829bb8d1fd9f8030f3b', 0, 0);

SET FOREIGN_KEY_CHECKS = 1;

🍊 配置文件修改

🍓 配置数据库信息: vi conf/application.yml
在这里插入图片描述
🍓 复制引擎服务中配置文件到web配置目录下面

[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/config/hazelcast-client.yaml apache-seatunnel-web-1.0.0-bin/conf/
[opensource@bigdata02 app]$ cp apache-seatunnel-2.3.3/connectors/plugin-mapping.properties apache-seatunnel-web-1.0.0-bin/conf/

在这里插入图片描述

🍊 配置下载数据源jar包

🍓 选择的mysql作为元数据库,对应的驱动包mysql-connector-java-8.0.25.jar放到libs
在这里插入图片描述
🍓 下载其它数据源资源包,首先下载源码,然后执行下载脚本download_datasource.sh(本地安装过git)

  • 🍒源码地址
    在这里插入图片描述
  • 🍒修改download_datasource.sh相关配置
    在这里插入图片描述
    在这里插入图片描述
  • 🍒执行download_datasource.sh
    在这里插入图片描述
  • 🍒开始下载jar包
    在这里插入图片描述
  • 🍒下载完成
    在这里插入图片描述
  • 🍒上传到libs目录
    在这里插入图片描述

🍊 启动服务

sh bin/seatunnel-backend-daemon.sh start

在这里插入图片描述

🍑四、web页面一览


🍊登录

🍓登录地址:http://xxxx:8801/ui/#/login ; 用户密码 admin/admin
在这里插入图片描述
在这里插入图片描述

🍊数据源

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

🍊任务

在这里插入图片描述
选不到数据源…
在这里插入图片描述
在这里插入图片描述
🍓尝试创建整库同步
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

🍊用户管理

在这里插入图片描述

🍊虚拟表

在这里插入图片描述

🍑五、其它


🍊问题记录

🍓服务启动失败:Application run failed org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor

14:29:08.032 [main] ERROR org.springframework.boot.SpringApplication - Application run failed
org.yaml.snakeyaml.scanner.ScannerException: while scanning an anchor
 in 'reader', line 30, column 15:
        password: &R7#J*vDVWEuT48m
                  ^
unexpected character found *(42)
 in 'reader', line 30, column 20:
        password: &R7#J*vDVWEuT48m
                       ^

        at org.yaml.snakeyaml.scanner.ScannerImpl.scanAnchor(ScannerImpl.java:1507)
        at org.yaml.snakeyaml.scanner.ScannerImpl.fetchAnchor(ScannerImpl.java:958)
        at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.java:389)
        at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:248)
        at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(ParserImpl.java:665)
        at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:165)
        at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:59)
        at org.yaml.snakeyaml.comments.CommentEventsCollector$1.peek(CommentEventsCollector.java:45)
        at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:140)
        at org.yaml.snakeyaml.comments.CommentEventsCollector.collectEvents(CommentEventsCollector.java:119)
        at org.yaml.snakeyaml.composer.Composer.composeScalarN

🍓解决办法,密码&R7#J*vDVWEuT48m增加双引号
在这里插入图片描述
在这里插入图片描述

🍊总结

  • 🍓apache-seatunnel-web-1.0.0-bin还处在开发早期阶段,并不成熟,功能也不完善
  • 🍓安装seatunnel的目的,就是想参考下web界面,寻求针对表的拖拽处理(减少sql编写),目前看效果不理想,也只是一个简单的输入、输出表字段映射,因此暂告一段落
    在这里插入图片描述
  • 24
    点赞
  • 35
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 8
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 8
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

韧小钊

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值