实验目的:
学习使用kettle8.2进行mongo数据库的同步方法
- 安装kettle8.2
参考网址
Kettle(六):centos7布署kettle8.2
https://blog.csdn.net/u010886217/article/details/102553677
1. 准备软件
pdi-ce-8.2.0.0-342.zip
jdk-8u202-linux-x64.tar.gz
2.安装各种依赖
yum -y install epel-release
wget http://li.nux.ro/download/nux/dextop/el7/x86_64/nux-dextop-release-0-5.el7.nux.noarch.rpm
rpm -ivh nux-dextop-release-0-5.el7.nux.noarch.rpm
yum install webkitgtk
yum install -y redhat-lsb
wget http://ftp.mozilla.org/pub/mozilla.org/xulrunner/nightly/2012/03/2012-03-02-03-32-11-mozilla-1.9.2/xulrunner-1.9.2.28pre.en-US.linux-x86_64.tar.bz2
tar -xjf xulrunner-1.9.2.28pre.en-US.linux-x86_64.tar.bz2
./xulrunner -register-global
3.配置过程
cd /opt
tar -zxvf jdk-8u202-linux-x64.tar.gz
unzip pdi-ce-8.2.0.0-342.zip
vim /etc/profile
export JAVA_HOME=/opt/jdk1.8.0_202
export JAVA_BIN=$JAVA_HOME/bin
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
source /etc/profile
cd /opt/data-integration
./kitchen.sh
./spoon.sh
4.遇到的问题说明
在一开始 使用的是kettle8.3 但是总是在最后执行spoon.sh的时报错
java.lang.NoClassDefFoundError: org/pentaho/vfs/ui/VfsResolver
后来,换成kettle8.2的版本后,成功
2.准备mongo环境
1.source
192.168.80.85 一个单节点mongo数据库
2.target
192.168.80.81 一个mongo复制集主库
3.目的
将85上的一个表或者一个库或者一些表同步到81上
3.kettle任务
保存任务名称
点击左上角的左三角号 执行
结果
2020/02/05 06:38:18 - Spoon - Using legacy execution engine
2020/02/05 06:38:18 - Spoon - Transformation opened.
2020/02/05 06:38:18 - Spoon - Launching transformation [85_2_81_test]...
2020/02/05 06:38:18 - Spoon - Started the transformation execution.
2020/02/05 06:38:18 - 85_2_81_test - Dispatching started for transformation [85_2_81_test]
2020/02/05 06:38:18 - MongoDB input.0 - Configuring connection with read preference: primary
2020/02/05 06:38:18 - MongoDB input.0 - No read preference tag sets defined
2020/02/05 06:38:18 - MongoDB input.0 - Configuring connection with write concern - w = 1, wTimeout = 0, journaled = false
2020/02/05 06:38:18 - MongoDB output.0 - Normal authentication for user admin
2020/02/05 06:38:18 - MongoDB output.0 - Configuring connection with read preference: primary
2020/02/05 06:38:18 - MongoDB output.0 - No read preference tag sets defined
2020/02/05 06:38:18 - MongoDB output.0 - Configuring connection with write concern - w = 1, wTimeout = 0, journaled = false
2020/02/05 06:38:18 - MongoDB input.0 - Query pulled data from: 192.168.80.85:27017
2020/02/05 06:38:19 - MongoDB input.0 - Finished processing (I=0, O=0, R=0, W=2294, U=0, E=0)
2020/02/05 06:38:20 - MongoDB output.0 - Finished processing (I=0, O=2294, R=2294, W=2294, U=0, E=0)
2020/02/05 06:38:20 - Spoon - The transformation has finished!!
查看结果