Sqoop 相关总结

1.sqoop命令的执行方式：

(1). Python : retCode = subprocess.call(sqoopCmd, shell=True)
eg:

    sqoopCmd = "sqoop import " \
               + "--connect '" + dbConnStr + "' " \
               + "--username '" + dbUser + "' " \
               + "--password '" + dbPass + "' " \
               + "--query \"" + sqoopQueryStr + "\" " \
               + "--null-string '\\\\N' " \
               + "--null-non-string '\\\\N' " \
               + "--target-dir '" + sqoopTargetFile + "' " \
               + "--hive-drop-import-delims " \
               + "--fields-terminated-by '" + sqoopFieldsSep + "' " \
               + "--lines-terminated-by '" + sqoopLinesSep + "' " \
               + "--append " \
               + "--temporary-rootdir '" + sqoopTmpDir + "' " \
               + "--m " + sqoopMaps \

    logging.info("sqoop cmd is : %s" % (sqoopCmd))
    retCode = subprocess.call(sqoopCmd, shell=True)

(2). Shell : 编写shell脚本，直接将参数写入shell中

eg:

#!/bin/bash

#CURR_DATE=`date +"%Y-%m-%d %H:%M:%S"`------>不能使用
v_sql="insert into origin_ennenergy_energytrade.test2 values('"$(date +"%Y-%m-%d %H:%M:%S")"','"Y"')"
echo $v_sql

#insert into origin_ennenergy_energytrade.test2 values('2016-08-09 10:39:44','Y')

hive -e "$v_sql;"
sqoop export --connect jdbc:mysql://ip:3306/test23?characterEncoding=utf8 --username root --password 123--table test2--export-dir /user/hive/warehouse/origin_ennenergy_energytrade.db/test2/* --input-fields-terminated-by "\t" --update-mode allowinsert --update-key times;

执行shell

sh test.sh

2.Sqoop参数详见：

http://sqoop.apache.org/docs/1.4.5/SqoopUserGuide.html