Sqoop 安装部署

1.获取安装包

[wzj@hadoop001 software]$ wget https://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.6-cdh5.16.2.tar.gz

2.解压缩

[wzj@hadoop001 software]$ tar -xvzf sqoop-1.4.6-cdh5.16.2.tar.gz -C …/app

软连接：

[wzj@hadoop001 software]$ ln -s sqoop-1.4.6-cdh5.16.2/ sqoop

3.配置环境变量并生效

[wzj@hadoop001 ~]$ vi .bahsrc

export SQOOP_HOME=/home/wzj/app/sqoop
export PATH=$SQOOP_HOME/bin:$PATH

[wzj@hadoop001 ~]$ source .bahsrc

4.配置文件

[wzj@hadoop001 conf]$ cp sqoop-env-template.sh sqoop-env.sh
-rw-rw-r–. 1 wzj wzj 3895 Jun 3 2019 oraoop-site-template.xml
-rwxr-xr-x. 1 wzj wzj 1400 Dec 24 10:39 sqoop-env.sh
-rwxr-xr-x. 1 wzj wzj 1345 Jun 3 2019 sqoop-env-template.sh
-rw-rw-r–. 1 wzj wzj 6044 Jun 3 2019 sqoop-site-template.xml
-rw-rw-r–. 1 wzj wzj 6044 Jun 3 2019 sqoop-site.xml
[wzj@hadoop001 conf]$ vi sqoop-env.sh

#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/home/wzj/app/hadoop

#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/home/wzj/app/hadoop

#Set the path to where bin/hive is available
export HIVE_HOME=/home/wzj/app/hive

5.json & mysql jdbc jar 包

$SQOOP_HOME/lib

[wzj@hadoop001 lib]$ pwd
/home/wzj/app/sqoop/lib
[wzj@hadoop001 lib]$ ll
-rw-r–r--. 1 wzj wzj 84697 Dec 24 09:46 java-json.jar
-rw-r–r--. 1 wzj wzj 1007502 Dec 24 09:46 mysql-connector-java-5.1.47.jar

6.Hive 配置文件

[wzj@hadoop001 conf]$ pwd
/home/wzj/app/hive/conf
[wzj@hadoop001 conf]$ cp hive-site.xml /home/wzj/app/sqoop/hive

[wzj@hadoop001 conf]$ pwd
/home/wzj/app/sqoop/conf
[wzj@hadoop001 conf]$ ll
total 32
-rw-r–r--. 1 wzj wzj 775 Dec 24 10:32 hive-site.xml
-rw-rw-r–. 1 wzj wzj 3895 Jun 3 2019 oraoop-site-template.xml
-rwxr-xr-x. 1 wzj wzj 1400 Dec 24 10:39 sqoop-env.sh
-rwxr-xr-x. 1 wzj wzj 1345 Jun 3 2019 sqoop-env-template.sh
-rw-rw-r–. 1 wzj wzj 6044 Jun 3 2019 sqoop-site-template.xml
-rw-rw-r–. 1 wzj wzj 6044 Jun 3 2019 sqoop-site.xml

部署完成！

7.检验

输入如下命令，列出mysql下所有的数据库
sqoop list-databases
–connect jdbc:mysql://ip:port
–password ****
–username root

[wzj@hadoop001 ~]$ sqoop list-databases \
> --connect jdbc:mysql://hadoop001:3306 \
> --password 123456 \
> --username root
19/12/24 12:56:36 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.16.2
19/12/24 12:56:36 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
19/12/24 12:56:36 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
information_schema
data_hive
mysql
performance_schema
test

输出与mysql数据库一致，则部署成功！