实时同步抽数ETL---Shell脚本

最新推荐文章于 2024-05-15 02:48:44 发布
最新推荐文章于 2024-05-15 02:48:44 发布
阅读量1k 收藏 11
点赞数
分类专栏： Shell编程 日常工作问题记录 文章标签： sqoop shell
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/cxl_shelly/article/details/116327128
版权

方式一（推荐）： 

sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  'test'  /user/hive/warehouse/s66_postgres.db/test_miaodazhuang1/dt=20210107  "SELECT CAST(EXTRACT(EPOCH FROM COALESCE(update_time,create_time)::TIMESTAMP WITH TIME ZONE)* 1000 AS int8)||'|9' AS event_timestamp,CAST('INSERT' AS VARCHAR) AS event_type,tt.* FROM test_miaodazhuang1 AS tt WHERE ((create_time >= '2021-01-07 00:00:00' and create_time<= '2021-01-08') OR (update_time >= '2021-01-07 00:00:00' and update_time<= '2021-01-08'))"  create_time  test_miaodazhuang1  root.etl

etl.sh 详细代码如下： 

#!/bin/bash
export LNAG=zh_CN.UTF-8

# 补数逻辑，pg => hdfs
# -- --schema <name>

connect=$1
username=$2
password=$3
hdfs_path=$4
query_condition=$5
alias_time=$6
tablename=$7
queue=$8

hdfs_path_tmp=${hdfs_path}_tmp
query_condition="${query_condition}"

echo "query_condition==> ${query_condition}"

sqoop_sql="sqoop import -D  mapred.job.queue.name=${queue}\
    --connect ${connect}?connectTimeout=300  \
    --username ${username}  \
    --password ${password}  \
    --target-dir ${hdfs_path_tmp}   \
    --fields-terminated-by '\t'  \
    --hive-delims-replacement ' '   \
    --null-string '\\\\N'  \
    --null-non-string '\\\\N' \
    --query \"${query_condition} AND \\\$CONDITIONS\" \
    --split-by ${alias_time} \
     "
echo "执行的sqoop脚本==> ${sqoop_sql}"
eval $sqoop_sql

if [ $? -eq 0 ] ;then
    echo "`date "+%Y-%m-%d %H:%M:%S"` ==> 提交抽数任务到 hdfs ${hdfs_path_tmp} 成功"

    echo "==>准备删除_SUCCESS文件"
    rm_sucess_file=" hdfs dfs -rm ${hdfs_path_tmp}/_SUCCESS "
    echo "rm_sucess_file == ${rm_sucess_file}"
    eval $rm_sucess_file
    if [ $? -eq 0 ] ;then
        echo "删除${rm_sucess_file}文件成功"
    else
        echo "删除${rm_sucess_file}文件失败"
    fi

    echo "==>判断${hdfs_path}路径是否存在"
    hdfs dfs -test -e ${hdfs_path}
    if [ $? -eq 0 ] ;then
        echo "${hdfs_path} exist"
    else
        echo "${hdfs_path} is not exist! create it"
        hdfs dfs -mkdir -p ${hdfs_path}
    fi
    
    echo "==>数据从临时路径${hdfs_path_tmp} 移动到${hdfs_path}"
        files=`hadoop fs -ls ${hdfs_path_tmp} | awk -F " " '{print $8}'`
        for file in ${files}
              do        
                        time_stamp=`date +%s`
                        filename=`echo ${file} | awk -F "part-m-" '{print $2}'`
                        mv_sql=" hdfs dfs -mv ${file} ${hdfs_path}/${tablename}-etl-${time_stamp}"
                        echo "mv_sql == ${mv_sql}"
                        eval $mv_sql
                        if [ $? -eq 0 ] ;then
                                echo "移动${file}到${hdfs_path}成功"
                        else
                                echo "移动${file}到${hdfs_path}失败"
                        fi
              done

    echo "==>删除临时文件夹${hdfs_path_tmp}"
    rm_tmp_dir="hdfs dfs -rm -r ${hdfs_path_tmp}"
    echo "rm_tmp_dir == ${rm_tmp_dir}"
    eval $rm_tmp_dir
    if [ $? -eq 0 ] ;then
        echo "删除临时文件夹${hdfs_path_tmp}成功"
    else
        echo "删除临时文件夹${hdfs_path_tmp}失败"
    fi
    echo "`date "+%Y-%m-%d %H:%M:%S"` ==> 抽数数据到 hdfs ${hdfs_path} 结束"
else
    echo "`date "+%Y-%m-%d %H:%M:%S"` ==> 抽数库中数据到 hdfs ${hdfs_path_tmp} 失败"
    exit 1
fi

生成的sqoop抽数的命令如下所示： 

sqoop import -D mapred.job.queue.name=root.etl	--connect jdbc:postgresql://192.168.0.36:5432/postgres?connectTimeout=300  	--username test       --password test  	--target-dir /user/hive/warehouse/s66_postgres.db/test_miaodazhuang1/dt=20210107_tmp   	--fields-terminated-by '\t'  	--hive-delims-replacement ' '   	--null-string '\\N'  	--null-non-string '\\N' 	--query "SELECT CAST(EXTRACT(EPOCH FROM COALESCE(update_time,create_time)::TIMESTAMP WITH TIME ZONE)* 1000 AS int8)||'|9' AS event_timestamp,CAST('INSERT' AS VARCHAR) AS event_type,tt.* FROM test_miaodazhuang1 AS tt WHERE ((create_time >= '2021-01-07 00:00:00' and create_time<= '2021-01-08') OR (update_time >= '2021-01-07 00:00:00' and update_time<= '2021-01-08')) AND \$CONDITIONS" 	--split-by create_time

方式二：

params_tmp.txt 

sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl
sh  /data/dfp/etl/etl.sh  jdbc:postgresql://192.168.0.36:5432/postgres  test  test  /user/hive/warehouse/s66_postgres.db/test_part/dt=20201209  "SELECT CASE WHEN update_time !=null THEN update_time ELSE create_time END as event_timestamp,'INSERT' AS event_type,tt.* FROM test_part AS tt WHERE ((create_time >= '2020-12-09 00:00:00' and create_time<= '2020-12-10') OR (update_time >= '2020-12-09 00:00:00' and update_time<= '2020-12-10')) AND \\\\\\$CONDITIONS"  create_time  test_part  root.etl

#!/bin/bash
export LNAG=zh_CN.UTF-8
# 补数逻辑，读取文件 array.txt=> line
cat params_tmp.txt|while read line

do
	echo "sqoop_sql==>${line}"
	eval $line

	if [ $? -eq 0 ] ;then
		echo "`date "+%Y-%m-%d %H:%M:%S"` ==> 提交抽数任务到 hdfs ${hdfs_path_tmp} 成功"
	else
		echo "`date "+%Y-%m-%d %H:%M:%S"` ==> 抽数库中数据到 hdfs ${hdfs_path_tmp} 失败"
		exit 1
	fi
	sleep 1
done

echo "==>准备删除_参数文件"
	rm_sucess_file="rm -f /data/dfp/etl/params_tmp.txt"
	echo "rm_sucess_file == ${rm_sucess_file}"
	eval $rm_sucess_file
echo "==>删除_参数文件成功!"

莱恩大数据
关注
  • 0
    点赞
  • 踩
  • 11
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
专栏目录
【etl】shell脚本实现mysql到dws的数据迁移
weixin_43990455的博客
03-15 345
mysql表需要每小时同步当天的数据到dws表。系统是linux，网络是专用网络，没权限安装其他etl工具。使用shell封装原始的数据库命令和linux命令完成数据迁移。
Linux下Shell编程实现基于Hadoop的ETL(导出篇)
Lucky的专栏
04-13 1030
conf/export.xml think_statistics bin/export.sh #!/bin/bash #get the environment if [ -f ~/.bashrc ]; then . ~/.bashrc fi #file exit if [ -z $1 ];then echo 'USAGE:COMMAND FILENAME' exit 0
大数据技术之shell
weixin_45077780的博客
06-11 383
文章目录第1章 Shell概述第2章 Shell解析器第1章 Shell概述第1章 Shell概述第1章 Shell概述第1章 Shell概述二级目录三级目录 第1章 Shell概述 大数据程序员为什么要学习Shell呢？ 1）需要看懂运维人员编写的Shell程序。 2）偶尔会编写一些简单Shell程序来管理集群、提高开发效率。 Shell是一个命令行解释器，它接收应用程序/用户命令，然后调用操作系统内核 Shell还是一个功能相当强大的编程语言，易编写、易调试、灵活性强 第2章 Shell解析器 第
【实时数仓篇】（02）基于 Flink 的典型 ETL 场景实现_买蓉 美团(3)
最新发布
2401_84586417的博客
05-15 416
本文是学习Flink中文社区实时数仓篇的总结，课程地址讲师：讲师：买蓉（美团点评 高级技术专家）。资源和运维成本较高，所以发展为kappa架构。
Linux下Shell编程实现基于Hadoop的ETL(分析篇)
Lucky的专栏
04-13 2065
conf/hive.xml create table if not exists think_statistics(date_type_name string,date_name string,type int,type_name string,count int) row format delimited fields terminated by '\\t' lines termi
Linux下Shell编程实现基于Hadoop的ETL(导入篇)
Lucky的专栏
04-13 1200
在这里我们直接给出代码，然后对代码解析分析来学习： conf/import.xml think_task think_task_child think_user_task think_user_task think_task bin/import.sh #!/bin/bash #get the environment if [ -f ~/.bash
基于Java的Porter数据同步中间件设计源码
04-08
用户可以通过编写自定义的Shell脚本来实现特定的数据转换或过滤逻辑，以适应各种复杂的数据同步场景。Shell脚本的引入降低了对接难度，使得Porter可以更方便地与其他系统进行交互。 在源码层面，我们可以看到项目中...
EDI shell脚本职责清单1
08-08
在这个场景中，我们看到一系列shell脚本被用来自动化处理数据处理流程，主要涉及Hadoop、Hive、MySQL和NLP（自然语言处理）的任务。以下是这些脚本的主要职责和涉及的技术点： 1. `/opt/running/edi/edi_new_in_...
aws-scripts:一个存储和共享我为管理AWS资源而开发的脚本的地方
03-30
10. 数据分析与处理：Shell脚本可以与AWS服务如Kinesis、Glue、EMR等配合，进行实时数据流处理、大数据分析或ETL（提取、转换、加载）任务。 "aws-scripts-main"可能包含了上述部分或全部功能的脚本实现，帮助用户...
Hadoop大数据开发实战-代码.rar
07-12
压缩包可能包含使用Shell脚本或Python进行数据处理的示例。 4. Hive数据仓库：Hive是基于Hadoop的数据仓库工具，用于数据ETL（提取、转换、加载）和查询。代码可能涉及创建表、导入数据、编写HQL（Hive Query ...
Distributed-Statistical-Computing:分布式统计计算的注释和分配
03-25
6. **Shell脚本**：在分布式计算中，Shell脚本用于自动化任务调度和资源管理。通过编写脚本，可以控制任务的启动、停止、监控等操作，也可以组合使用各种工具进行复杂的数据处理流程。 7. **统计分析库**：在分布式...
基于Hadoop的ETL处理Shell架构5
04-14
基于Hadoop的ETL大数据处理流程，配置方便，完全Shell编程实现，支持Bash Shell环境的Linux系统。
全量数据同步ETL脚本案例.zip
12-11
对源数据进行全部抽取到目的源上，相当于将文件完全复制一份到目的地，该方式可以完全保证数据的一致性问题
ETL开发常用shell文件操作
china_demon的专栏
09-07 125
【代码】ETL开发常用shell文件操作。
Linux下Shell编程实现基于Hadoop的ETL(流程篇)
Lucky的专栏
04-13 1690
conf/flow.sh ../conf/import.xml ../conf/hive.xml ../conf/export.xml bin/flow.sh #! /bin/bash #get the environment if [ -f ~/.bashrc ]; then . ~/.bashrc fi #file exit if [ -z $1 ];then ech
ETL 脚本任务
weixin_33682790的博客
09-04 130
Public Sub Main() ' ' Add your code here ' Dim sbWA As New StringBuilder Dim i As Integer Dim arWA As New Collections.ArrayList Dim bEOF As Boole...
etl java脚本_ETL Pentaho代码学习笔记
weixin_29828857的博客
02-26 208
1、通过设置KETTLE_HOME环境变量可以让.kettle不需要在user.home下2、默认kettle_home 为User.home，如果要自定义需要设置环境变量KETTLE_HOME3、在.kettle目录下可放置以下文件或目录：文件名说明kettle.properties内部运行时的环境变量.languageChoice设置运行的语言，方便进行界面语言的翻译--内容：LocaleDe...
BI-Pilot ETL：数据仓库的核心工具
BI-Pilot ETL产品是博易智软（北京）技术股份有限公司推出的一款数据处理工具，主要用于数据仓库（Data Warehouse，简称DW）和商业智能（Business Intelligence，简称BI）领域。该产品旨在解决数据抽取、清洗、转换...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

莱恩大数据

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值