Sqoop： 环境搭建、实例及概念

简介

Sqoop是一个用于在外部结构化数据与Hadoop之间导入导出数据的工具。

Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases.

Sqoop:

• 是一个命令行工具？
• 使得结构化数据与Hadoop之间的导入导出变得容易
• 可以导入整个库或者单个表
• 可以与Oozie集成，将 导入导出作为工作流的一部分。
• 内部转化为MapReduce，用于传输数据

下载安装配置

根据Hadoop版本选择对应的Sqoop，我这里是Hadoop，选择Sqoop 1.99.6：

wget http://mirrors.hust.edu.cn/apache/sqoop/1.99.6/sqoop-1.99.6-bin-hadoop200.tar.gz

tar zxvf sqoop-1.99.6-bin-hadoop200.tar.gz 

mv sqoop-1.99.6-bin-hadoop200 /home/sqoop-1.99.6

配置PATH：

vim /etc/profile
export SQOOP_HOME=/home/sqoop-1.99.6
export PATH=$PATH:$SQOOP_HOME/bin

source /etc/profile

配置server/conf/catalina.properties

common.loader=${
   catalina.base}/lib,${
   catalina.base}/lib/*.jar,${
   catalina.home}/lib,${
   catalina.home}/lib/*.jar,${
   catalina.home}/../lib/*.jar,/home/hadoop-2.6.0/share/hadoop/common/*.jar,/home/hadoop-2.6.0/share/hadoop/common/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/hdfs/*.jar,/home/hadoop-2.6.0/share/hadoop/hdfs/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/mapreduce/*.jar,/home/hadoop-2.6.0/share/hadoop/mapreduce/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/yarn/*.jar,/home/hadoop-2.6.0/share/hadoop/yarn/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/tools/*.jar,/home/hadoop-2.6.0/share/hadoop/tools/lib/*.jar,/home/hadoop-2.6.0/share/hadoop/httpfs/tomcat/lib/*.jar 

配置server/conf/sqoop.properties:

org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/home/hadoop-2.6.0/etc/hadoop

验证是否配置正确：

sqoop2-tool veriify

启动sqoop serv