cdh sqoop 配置_Sqoop-1.4.6安装部署及详细使用介绍

之所以选择Sqoop1是因为Sqoop2目前问题太多。无法正常使用，综合比较后选择Sqoop1。

Sqoop1安装配置比较简单

一、安装部署

(1)、下载安装包

解压到/home/duanxz/sqoop

tar -zxvf sqoop-1.4.6-cdh5.5.2.tar.gz

(2)、拷贝mysql的jdbc驱动包mysql-connector-java-5.1.31-bin.jar到sqoop/lib目录下。

duanxz@three:~/sqoop/sqoop-1.4.6-cdh5.5.2/lib$ ll mysql-connector-java-5.1.31.jar-rw------- 1 duanxz duanxz 964879 Jun 19 08:22 mysql-connector-java-5.1.31.jar

duanxz@three:~/sqoop/sqoop-1.4.6-cdh5.5.2/lib$

(3)、配置环境变量

#sqoop

export SQOOP_HOME=/home/duanxz/sqoop/sqoop-1.4.6-cdh5.5.2export PATH="$PATH:$JAVA_HOME/bin:$HIVE_HOME/bin:$HIVE_HOME/conf:$SQOOP_HOME/bin"

(4)、复制sqoop/conf/sqoop-env-template.sh为sqoop-env.sh

添加相关的配置

#Set path to where bin/hadoop isavailable

export HADOOP_COMMON_HOME=/usr/local/hadoop-2.7.6#Set path to where hadoop-*-core.jar isavailable

export HADOOP_MAPRED_HOME=/usr/local/hadoop-2.7.6#set the path to where bin/hbase isavailable

#export HBASE_HOME=#Set the path to where bin/hive isavailable

export HIVE_HOME=/home/duanxz/hive/apache-hive-2.1.1-bin

#Set the path for where zookeper config dir is#export ZOOCFGDIR=

(5)、测试Sqoop

sqoop help

结果：

duanxz@ubuntu:~/sqoop/sqoop-1.4.6-cdh5.5.2/bin$ sqoop help

Warning:/home/duanxz/sqoop/sqoop-1.4.6-cdh5.5.2/../hbase does notexist! HBase imports will fail.

Pleaseset $HBASE_HOME to the root ofyour HBase installation.

Warning:/home/duanxz/sqoop/sqoop-1.4.6-cdh5.5.2/../hcatalog does notexist! HCatalog jobs will fail.

Pleaseset $HCAT_HOME to the root ofyour HCatalog installation.

Warning:/home/duanxz/sqoop/sqoop-1.4.6-cdh5.5.2/../accumulo does notexist! Accumulo imports will fail.

Pleaseset $ACCUMULO_HOME to the root ofyour Accumulo installation.

Warning:/home/duanxz/sqoop/sqoop-1.4.6-cdh5.5.2/../zookeeper does notexist! Accumulo imports will fail.

Pleaseset $ZOOKEEPER_HOME to the root ofyour Zookeeper installation.18/06/19 18:20:23 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.5.2usage: sqoop COMMAND[ARGS]Available commands:

codegen Generate codeto interact with databaserecordscreate-hive-table Import a table definition intoHive

eval Evaluate a SQL statementanddisplay the results

export Export an HDFS directoryto a database tablehelp List available commands

import Import atable from a database toHDFS

import-all-tables Import tables from a database toHDFS

import-mainframe Import datasets from a mainframe server toHDFS

jobWork withsaved jobs

list-databases List available databases ona server

list-tables List available tables in a databasemerge Merge resultsofincremental imports

metastore Run a standalone Sqoop metastore

version Display version information

See'sqoop help COMMAND' for information ona specific command.

duanxz@ubuntu:~/sqoop/sqoop-1.4.6-cdh5.5.2/bin$

说明：因为我们没有基于hadoop安装HBase，所以HBase相关的命令不能用，但是操作hadoop分布式文件系统的命令是可以用的。

再次执行sqoop version

duanxz@ubuntu:~/sqoop/sqoop-1.4.6-cdh5.5.2/bin$ sqoop version

Warning:/home/duanxz/sqoop/sqoop-1.4.6-cdh5.5.2/../hbase does notexist! HBase imports will fail.

Pleaseset $HBASE_HOME to the root ofyour HBase installation.

Warning:/home/duanxz/sqoop/sqoop-1.4.6-cdh5.5.2/../hcatalog does notexist! HCatalog jobs will fail.

Pleaseset $HCAT_HOME to the root ofyour HCatalog installation.

Warning:/home/duanxz/sqoop/sqoop-1.4.6-cdh5.5.2/../accumulo does notexist! Accumulo imports will fail.

Pleaseset $ACCUMULO_HOME to the root ofyour Accumul