kettle 使用 连接mongodb ,报错处理
com.mongodb.MongoCommandException: Command failed with error 16436: 'Unrecognized pipeline stage name: 'ADD_TIME'' on server 192.168.2.144:27017. The full response is { "ok" : 0.0, "errmsg" : "Unrecognized pipeline stage name: 'ADD_TIME'", "code" : 16436 }
Command failed with error 16436: 'Unrecognized pipeline stage name: 'ADD_TIME'' on server 192.168.2.144:27017. The full response is { "ok" : 0.0, "errmsg" : "Unrecognized pipeline stage name: 'ADD_TIME'", "code" : 16436 }
at org.pentaho.di.trans.steps.mongodbinput.MongoDbInput.processRow(MongoDbInput.java:137)
at org.pentaho.di.trans.step.RunThread.run(RunThread.java:62)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.mongodb.MongoCommandException: Command failed with error 16436: 'Unrecognized pipeline stage name: 'ADD_TIME'' on server 192.168.2.144:27017. The full response is { "ok" : 0.0, "errmsg" : "Unrecognized pipeline stage name: 'ADD_TIME'", "code" : 16436 }
先借鉴以下这篇文章
http://www.spring4all.com/article/19550
未解决问题,所有尝试其他配置(注意 上面参考文章和本篇解决问题的根本区别是 replica 模式)
取消 use all replica set members/mongos 选项
然后在query选项卡中,取消 query is aggregation pipeline
再测,通过测试,可以正常下载数据了
其他页面参考如下图:
---- 关于 kettle 参数传入的方式记录
#!/bin/bash
set -e
basepath="$( cd "$( dirname "$0" )" && pwd )"
source /etl/etl-fr/setting/parArg.sh $@
source /etl/etl-fr/setting/kettle.confif [ -z "${startDate}" ]; then
echo "ERROR: startDate is lost"
exit -1
fiif [ -z "${endDate}" ]; then
echo "ERROR: endDate is lost"
exit -1
fiecho $startDate
echo $endDate
$KETTLE -rep kettle -job data_check/JOB_ORDER_CHECK -param:startDate="$startDate" -param:endDate="$endDate" | tee -a $KETTLE_LOG/job_a01_order_check_month.log`date +"%Y%m%d" -d "-0 day"` 2>&1
在所被调起的job中需要配置 “命名参数”
然后便可以在job中任意位置使用变量了,比如如下的sql中:(注意 使用变量替换的选项需要选中)
当然,根据传入查数,生成更多的参数也是非常有必要的,可以在job的靠前位置,使用tran来生成多个参数
结构如下:
getdate阶段,现在过去job的传入参数
然后在script中,计算各种变变量,并发布字段
在最后阶段进行变量的job内全局发布