sqoop 部署安装

1. 下载，上传，解压

下载

• 下载的sqoop版本要与环境的hadoop版本相匹配，本文使用的版本是：
  sqoop-1.4.6-cdh5.7.1.tar.gz

上传

• [hadoop@wjxhadoop001 software]$ rz ##可以使用wget

解压

    1. [hadoop@wjxhadoop001 software]$ tar -xzvf sqoop-1.4.6-cdh5.7.1.tar.gz

    2. [hadoop@wjxhadoop001 software]$ sudo chown -R hadoop:hadoop sqoop-1.4.6-cdh5.7.1

    4. [hadoop@wjxhadoop001 software]$ sudo mv sqoop-1.4.6-cdh5.7.1 sqoop

拷贝mysql驱动包

• 拷贝mysql的jdbc驱动包mysql-connector-java-5.1.45-bin.jar到sqoop/lib目录下

  1. [hadoop@wjxhadoop001 lib]$ sudo cp /opt/software/hive/lib/mysql-connector-java-5.1.45-bin.jar .
  2. hive目录下已下载好驱动包
  

配置环境变量

• 将sqoop home配置到环境变量，方便使用

  1. [hadoop@wjxhadoop001sqoop]$ sudo vi /etc/profile
  2. export SQOOP_HOME=/opt/software/sqoop 
  3. export PATH=$PATH:$SQOOP_HOME/bin
  

配置sqoop-env.sh

• copy sqoop/conf/sqoop-env-template.sh 为sqoop-env.sh
  并进行配置

  1. [hadoop@wjxhadoop001 conf]$ cp sqoop-env-template.sh sqoop-env.sh
  
  2. #Setpath to where bin/hadoop is available
  3. exportHADOOP_COMMON_HOME=/opt/software/hadoop
  4. #Setpath to where hadoop-*-core.jar isavailable
  5. exportHADOOP_MAPRED_HOME=/opt/software/hadoop
  6. #setthe path to where bin/hbase isavailable
  7. #exportHBASE_HOME=/opt/software/hbase 
  8. #Setthe path to where bin/hive is available
  9. exportHIVE_HOME= /opt/software/hive
  10. #Setthe path for where zookeper config diris
  11. #exportZOOCFGDIR= /opt/software/zookeeper
  

查看版本

    1. [hadoop@wjxhadoop001 conf]$ sqoop version
    2. Warning: /opt/software/sqoop/../hbase does not exist! HBase imports will fail.
    3. Please set $HBASE_HOME to the root of your HBase installation.
    4. Warning: /opt/software/sqoop/../hcatalog does not exist! HCatalog jobs will fail.
    5. Please set $HCAT_HOME to the root of your HCatalog installation.
    6. Warning: /opt/software/sqoop/../accumulo does not exist! Accumulo imports will fail.
    7. Please set $ACCUMULO_HOME to the root of your Accumulo installation.
    8. Warning: /opt/software/sqoop/../zookeeper does not exist! Accumulo imports will fail.
    9. Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
    10. 18/01/14 12:47:03 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.7.1
    11. Sqoop 1.4.6-cdh5.7.1
    12. git commit id 
    13. Compiled by jenkins on Wed Mar 23 11:30:51 PDT 2016

    14. [hadoop@wjxhadoop001 conf]$ sqoop import --help

来自@若泽大数据