Sqoop安装配置教程

1.安装部署
（1）、下载地址：http://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.6-cdh5.5.2.tar.gz
解压到／home/neil/Downloads/hadoop-2.7.3/sqoop-1.4.6
（2）、拷贝MySQL的jdbc驱动包mysql-connector-java-5.1.31-bin.jar到sqoop/lib目录下。
（3）、配置环境变量

sqoop

export SQOOP_HOME=／home/neil/Downloads/hadoop-2.7.3/sqoop-1.4.6
export PATH=$PATH:$SQOOP_HOME/bin
（4）、复制sqoop/conf/sqoop-env-template.sh为sqoop-env.sh
添加相关的配置

Setpath to where bin/hadoop is available

exportHADOOP_COMMON_HOME=／home/neil/Downloads/hadoop-2.7.3

Setpath to where hadoop-*-core.jar isavailable

exportHADOOP_MAPRED_HOME=／home/neil/Downloads/hadoop-2.7.3

setthe path to where bin/hbase isavailable

exportHBASE_HOME=/opt/cdh5/hbase-1.1.8

Setthe path to where bin/hive is available

exportHIVE_HOME= ／home/neil/Downloads/hadoop-2.7.3/hive-1.2.1

Setthe path for where zookeper config diris

exportZOOCFGDIR= ／home/neil/Downloads/hadoop-2.7.3/zookeeper-3.4.5
（5）测试Sqoop version
发现有警告
修改$SQOOP_HOME/bin/configure-sqoop
注释掉HCatalog，Accumulo检查(除非你准备使用HCatalog，Accumulo等HADOOP上的组件)

Moved to be a runtime check in sqoop.

if [ ! -d “${HCAT_HOME}” ]; then

echo “Warning: $HCAT_HOME does not exist! HCatalog jobs willfail.”

echo ‘Please set $HCAT_HOME to the root of your HCatalog installation.’

fi

if [ ! -d “${ACCUMULO_HOME}” ];then

echo “Warning: $ACCUMULO_HOME does not exist! Accumulo imports willfail.”

echo ‘Please set $ACCUMULO_HOME to the root of your Accumuloinstallation.’

fi

再次执行sqoop version
你也可以查看某一个命令的使用说明：
$sqoopimport –help$ sqoophelp import

sqoop import 的一个示例如下：
登录MYSQL
1.mysql -u root -p

2.创建数据库hadoopguide;
create database hadoopguide;

use hadoopguide;

3.创建表
create table widgets(id int not null primary key auto_increment,widget_name varchar(64) not null,price decimal(10,2),design_date date,version int,design_comment varchar(100));

4.插入几条记录
insert into widgets values(null,’sprocket’,0.25,’2010-02-10’,1,’connects two gizmos’);
insert into widgets values(null,’gizmo’,4.00,’2009-02-10’,1,null);
insert into widgets values(null,’sprocket’,99.99,’2011-02-10’,1,’Our flagship product’);

5.退出
quit

6.使用sqoop导入这个表数据到HDFS

$ sqoop import –connect jdbc:mysql://localhost:3306/hadoopguide –username root –password neil –table widgets

你还可以使用 –options-file 来传入一个文件，使用这种方式可以重用一些配置参数：
$ sqoop –options-file /users/homer/work/import.txt –table TEST
/users/homer/work/import.txt 文件内容如下：
import
–connect
jdbc:mysql://192.168.56.121:3306/metastore
–username
hiveuser
–password
redhat