sqoop1.4.6 java_Sqoop(sqoop-1.4.6) 安装使用

2、安装部署

(1) 解压：

sudo tar -zxvf sqoop-1.4.6.bin__hadoop-1.0.0.tar.gz -C /home/sqoop/

sudo mv sqoop-1.4.6.bin__hadoop-1.0.0/ sqoop-1.4.6

(2) 设置环境变量(/etc/profile 或~/.bashrc)

#Java

export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_80

export JRE_HOME=${JAVA_HOME}/jre

export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib

export PATH=${JAVA_HOME}/bin:$PATH

#ZK

export ZK_HOME=/home/zookeeper/zookeeper-3.4.10

export PATH=$ZK_HOME/bin:$PATH

#set hadoop environment

export HADOOP_HOME=/home/hadoop/hadoop-home/hadoop-2.8.0/

export PATH=${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:$PATH

# Hive

export HIVE_HOME=/home/hive/apache-hive-2.1.1/

export HIVE_CONF_DIR=$HIVE_HOME/conf

export CLASSPATH=$CLASSPATH:$HIVE_HOME/lib

export PATH=$PATH:$HIVE_HOME/bin

#set sqoop environment

export SQOOP_HOME=/home/sqoop/sqoop-1.4.6

export PATH=$SQOOP_HOME/bin:$PATH

#export SQOOP_SERVER_EXTRA_LIB=/home/sqoop/sqoop-1.4.6/extra

(3) 复制jar包

hadoop-common-2.8.0.jar，mysql-connector-java-5.1.42-bin.jar，sqoop-1.4.6.jar

cp /home/hadoop/hadoop-home/hadoop-2.8.0/share/hadoop/common/hadoop-common-2.8.0.jar sqoop-1.4.6/lib/

cp ~/Downloads/mysql-connector-java-5.1.42/mysql-connector-java-5.1.42-bin.jar sqoop-1.4.6/lib/

(4)修改配置文件sqoop-env.sh sqoop-site.xml

hadoop@hadoop01:/home/sqoop/sqoop-1.4.6/conf$ cp sqoop-env-template.sh sqoop-env.sh

cp sqoop-env-template.sh sqoop-env.sh

vi sqoop-env.sh

# Set Hadoop-specific environment variables here.

#Set path to where bin/hadoop is available

export HADOOP_COMMON_HOME=/home/hadoop/hadoop-home/hadoop-2.8.0/

#Set path to where hadoop-*-core.jar is available

export HADOOP_MAPRED_HOME=/home/hadoop/hadoop-home/hadoop-2.8.0/

#set the path to where bin/hbase is available

export HBASE_HOME=/home/hbase/hbase-1.2.5/

#Set the path to where bin/hive is available

export HIVE_HOME=/home/hive/apache-hive-2.1.1/

#Set the path for where zookeper config dir is

export ZOOCFGDIR=/home/zookeeper/zookeeper-3.4.10/conf/

vi sqoop-site.xml

sqoop.metastore.client.autoconnect.url

jdbc:hsqldb:file:/home/sqoop/sqoop-meta/meta.db;shutdown=true

jdbc:mysql://hadoop04/huntsqoop?useUnicode=true&characterEncoding=utf-8

sqoop.metastore.client.autoconnect.username

sqoop

The username to bind to the metastore.

sqoop.metastore.client.autoconnect.password

sqoop

The password to bind to the metastore.

(5)测试：

sqoop create-hive-table --connect jdbc:mysql://172.16.0.111/test --username test--password test --table test01 --hive-table test01 --fields-terminated-by ',' --hive-overwrite

sqoop list-tables --connect jdbc:mysql://172.16.0.111/test --username test --password test

sqoop import --connect jdbc:mysql://172.16.0.111:3306/test --username test--password test123456--table test01--hive-import --hive-table test01 -m 1

sqoop import --connect jdbc:mysql://172.16.0.196:3306/lfdbcf --username myjht --password jht123456 --table htrelation --hive-table htrelation --input-fields-terminated-by ',' --hive-import --hive-overwrite --null-string '\\N' --null-non-string '\\N'