部署SeaTunnel单节点Standalone 模式环境

1.前置准备:

        SeaTunnel支持运行在JDK8及以上环境。该步骤需要用户自行安装JDK环境。

2.下载安装包

        本次部署使用的是2.3.1版本。如果你需要下载其它版本，可以从如下网址中查询对应的版本。

Apache SeaTunnel

​
mkdir ~/seatunnel
cd ~/seatunnel

wget https://dlcdn.apache.org/seatunnel/2.3.1/apache-seatunnel-incubating-2.3.1-bin.tar.gz

tar -zxvf apache-seatunnel-incubating-2.3.1-bin.tar.gz

​

3.安装需要的插件

        SeaTunnel需要自己安装同步数据需要的连接器插件，需要用户编辑config下plugin_config文件，该文件描述了需要下载和安装的连接器插件，默认所有已经支持的连接器插件都会下载和安装。我们可以修改该文件，删除我们不需要的插件，只保留我们需要的插件。

cd ~/seatunnel/apache-seatunnel-incubating-2.3.1
vi config/plugin_config

4.运行下载命令

        运行下载安装连接器插件的命令：

cd ~/seatunnel/apache-seatunnel-incubating-2.3.1
sh bin/install-plugin.sh

等待命令执行完成，连接器插件即下载安装完成。安装完成后可见~/seatunnel/apache-seatunnel-incubation-2.3.1/connectors/seatunnel/目录下已经有了安装好的连接器插件。

-rw-r--r-- 1 root root  11501046 10月 24 19:00 connector-amazondynamodb-2.3.1.jar
-rw-r--r-- 1 root root     70099 10月 24 19:01 connector-assert-2.3.1.jar
-rw-r--r-- 1 root root  13850207 10月 24 19:17 connector-cassandra-2.3.1.jar
-rw-r--r-- 1 root root  31837415 10月 24 19:39 connector-cdc-mysql-2.3.1.jar
-rw-r--r-- 1 root root  30487309 10月 24 20:17 connector-clickhouse-2.3.1.jar
-rw-r--r-- 1 root root     52023 2月  18 2022 connector-console-2.3.1.jar
-rw-r--r-- 1 root root   7004361 10月 24 20:29 connector-datahub-2.3.1.jar
-rw-r--r-- 1 root root   5575890 10月 24 20:33 connector-dingtalk-2.3.1.jar
-rw-r--r-- 1 root root   1972948 10月 24 20:36 connector-doris-2.3.1.jar
-rw-r--r-- 1 root root   5475697 10月 24 20:43 connector-elasticsearch-2.3.1.jar
-rw-r--r-- 1 root root    730013 10月 24 20:44 connector-email-2.3.1.jar
-rw-r--r-- 1 root root    153432 2月  18 2022 connector-fake-2.3.1.jar
-rw-r--r-- 1 root root  23578424 10月 24 22:10 connector-file-hadoop-2.3.1.jar
-rw-r--r-- 1 root root  23574827 10月 24 22:57 connector-file-local-2.3.1.jar
-rw-r--r-- 1 root root 235391594 10月 25 00:36 connector-file-oss-jindo-2.3.1.jar
-rw-r--r-- 1 root root  26585017 10月 25 00:39 connector-file-s3-2.3.1.jar
-rw-r--r-- 1 root root  23878459 10月 25 00:43 connector-file-sftp-2.3.1.jar
-rw-r--r-- 1 root root   6840166 10月 25 00:52 connector-google-sheets-2.3.1.jar
-rw-r--r-- 1 root root  50730916 10月 25 03:22 connector-hbase-2.3.1.jar
-rw-r--r-- 1 root root  23598219 10月 25 01:05 connector-hive-2.3.1.jar
-rw-r--r-- 1 root root   5177581 10月 25 01:12 connector-http-base-2.3.1.jar
-rw-r--r-- 1 root root   5180838 10月 25 01:19 connector-http-feishu-2.3.1.jar
-rw-r--r-- 1 root root   5186017 10月 25 01:27 connector-http-github-2.3.1.jar
-rw-r--r-- 1 root root   5185488 10月 25 01:22 connector-http-gitlab-2.3.1.jar
-rw-r--r-- 1 root root   5184487 10月 25 01:32 connector-http-jira-2.3.1.jar
-rw-r--r-- 1 root root   5185674 10月 25 01:36 connector-http-klaviyo-2.3.1.jar
-rw-r--r-- 1 root root   5185305 10月 25 01:41 connector-http-lemlist-2.3.1.jar
-rw-r--r-- 1 root root   5188184 10月 25 01:44 connector-http-myhours-2.3.1.jar
-rw-r--r-- 1 root root   5185496 10月 25 01:48 connector-http-notion-2.3.1.jar
-rw-r--r-- 1 root root   5185558 10月 25 01:52 connector-http-onesignal-2.3.1.jar
-rw-r--r-- 1 root root   5185309 10月 25 01:54 connector-http-wechat-2.3.1.jar
-rw-r--r-- 1 root root 168487797 10月 25 02:45 connector-hudi-2.3.1.jar
-rw-r--r-- 1 root root  29383607 10月 25 02:49 connector-iceberg-2.3.1.jar
-rw-r--r-- 1 root root   3444329 10月 25 02:49 connector-influxdb-2.3.1.jar
-rw-r--r-- 1 root root   5780985 10月 25 02:50 connector-iotdb-2.3.1.jar
-rw-r--r-- 1 root root    288042 10月 25 02:50 connector-jdbc-2.3.1.jar
-rw-r--r-- 1 root root  13731460 10月 25 02:53 connector-kafka-2.3.1.jar
-rw-r--r-- 1 root root  25520058 10月 25 02:54 connector-kudu-2.3.1.jar
-rw-r--r-- 1 root root  20540761 10月 25 02:55 connector-maxcompute-2.3.1.jar
-rw-r--r-- 1 root root   2419398 10月 25 02:56 connector-mongodb-2.3.1.jar
-rw-r--r-- 1 root root   5070244 10月 25 02:57 connector-neo4j-2.3.1.jar
-rw-r--r-- 1 root root 148797758 10月 25 03:11 connector-openmldb-2.3.1.jar
-rw-r--r-- 1 root root  42273251 10月 25 03:11 connector-pulsar-2.3.1.jar
-rw-r--r-- 1 root root    790519 10月 25 03:11 connector-rabbitmq-2.3.1.jar
-rw-r--r-- 1 root root   1331781 10月 25 03:12 connector-redis-2.3.1.jar
-rw-r--r-- 1 root root  35638134 10月 25 03:16 connector-s3-redshift-2.3.1.jar
-rw-r--r-- 1 root root   1623004 10月 25 03:21 connector-selectdb-cloud-2.3.1.jar
-rw-r--r-- 1 root root    625274 10月 25 03:16 connector-sentry-2.3.1.jar
-rw-r--r-- 1 root root   5930367 10月 25 03:17 connector-slack-2.3.1.jar
-rw-r--r-- 1 root root    134855 10月 25 03:17 connector-socket-2.3.1.jar
-rw-r--r-- 1 root root  20267439 10月 25 03:20 connector-starrocks-2.3.1.jar
-rw-r--r-- 1 root root  10758366 10月 25 03:21 connector-tablestore-2.3.1.jar

5.启动一个单节点SeaTunnel

cd ~/seatunnel/apache-seatunnel-incubating-2.3.1
nohup sh bin/seatunnel-cluster.sh 2>&1 &

通过jps命令我们能查看进程是否已经启动，进程名为SeaTunnelServer

6.使用自带例子测试任务

        在config目录下有一个自带的实时同步任务的配置文件v2.streaming.conf.template，该文件定义了一个作业，使用一个叫FakeSource的Source连接器生成数据，并将数据发送给Console这个Sink，Console Sink的作用是将接收到的数据打印到控制台。

        所以运行该作业可以看到在控制台中会打印数据，因为是实时作业，该作业不会自动停止，v2.streaming.conf.template文件的内容如下:

#
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#    http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
######
###### This config file is a demonstration of streaming processing in Seatunnel config
######

env {
  # You can set flink configuration here
  execution.parallelism = 2
  job.mode = "STREAMING"
  checkpoint.interval = 2000
  #execution.checkpoint.interval = 10000
  #execution.checkpoint.data-uri = "hdfs://localhost:9000/checkpoint"
}

source {
  # This is a example source plugin **only for test and demonstrate the feature source plugin**
  FakeSource {
    parallelism = 2
    result_table_name = "fake"
    row.num = 16
    schema = {
      fields {
        name = "string"
        age = "int"
      }
    }
  }

  # If you would like to get more information about how to configure Seatunnel and see full list of source plugins,
  # please go to https://seatunnel.apache.org/docs/category/source-v2
}

sink {
  Console {
  }

  # If you would like to get more information about how to configure Seatunnel and see full list of sink plugins,
  # please go to https://seatunnel.apache.org/docs/category/sink-v2

         执行该demo作业：

cd ~/seatunnel/apache-seatunnel-incubating-2.3.1
sh bin/seatunnel.sh --config config/v2.streaming.conf.template

        运行结果

***********************************************
Job Id                    :  769365937475289089
Read Count So Far         :                  32
Write Count So Far        :                  32
Average Read Count        :                 0/s
Average Write Count       :                 0/s
Last Statistic Time       : 2023-10-25 09:23:34
Current Statistic Time    : 2023-10-25 09:24:34
***********************************************

2023-10-25 09:25:33,998 INFO  org.apache.seatunnel.engine.client.job.JobMetricsRunner - 
***********************************************
           Job Progress Information
***********************************************
Job Id                    :  769365937475289089
Read Count So Far         :                  32
Write Count So Far        :                  32
Average Read Count        :                 0/s
Average Write Count       :                 0/s
Last Statistic Time       : 2023-10-25 09:24:34
Current Statistic Time    : 2023-10-25 09:25:33
***********************************************

        说明作业正常运行，通过Control+C结束该作业，中止作业运行,SeaTunnel部署及验证完成。