Sqoop1.4.5（基于Hadoop2.2环境）的安装测试部署

特别说明：sqoop-1.4.5.bin__hadoop-0.20.tar.gz不支持hadoop 2.2版本

1、下载安装包sqoop-1.4.5.bin__hadoop-2.0.4-alpha.tar.gz

2、解压缩：tar -zxvf sqoop-1.4.5.bin__hadoop-2.0.4-alpha.tar.gz

3、配置环境变量

export HADOOP_COMMON_HOME=/root/install/hadoop-2.2.0
export HADOOP_MAPRED_HOME=/root/install/hadoop-2.2.0/share/hadoop/mapreduce 
export HBASE_HOME=/root/install/hbase-0.96.2
export HIVE_HOME=/root/install/apache-hive-0.13.1-bin

4、在$SQOOP_HOME/conf目录下新增sqoop-env.sh文件，其内容如下

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

# included in all the hadoop scripts with source command
# should not be executable directly
# also should not be passed any arguments, since we need original $*

# Set Hadoop-specific environment variables here.

#Set path to where bin/hadoop is available
#export HADOOP_COMMON_HOME=

#Set path to where hadoop-*-core.jar is available
#export HADOOP_MAPRED_HOME=

#set the path to where bin/hbase is available
#export HBASE_HOME=

#Set the path to where bin/hive is available
#export HIVE_HOME=

#Set the path for where zookeper config dir is
#export ZOOCFGDIR=
export HADOOP_COMMON_HOME=/root/install/hadoop-2.2.0
export HADOOP_MAPRED_HOME=/root/install/hadoop-2.2.0/share/hadoop/mapreduce
export HBASE_HOME=/root/install/hbase-0.96.2
export HIVE_HOME=/root/install/apache-hive-0.13.1-bin

5、把mysql-connector-java-5.1.7-bin.jar放到/root/install/sqoop-1.4.5/lib目录下

6、测试链接mysql是否正常

sqoop list-databases --connect jdbc:mysql://IP:3306/ --username root --password 123456

Warning: /root/install/sqoop-1.4.5/../hcatalog does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /root/install/sqoop-1.4.5/../accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
Warning: /app/zookeeper-3.4.5 does not exist! Accumulo imports will fail.
Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.
14/09/26 11:18:54 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
14/09/26 11:18:54 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
14/09/26 11:18:55 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
information_schema
hive
mysql
test

问题总结:

在java中调用sqoop接口进行mysql和hdfs直接数据传输时，遇到以下错误：

Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

这里需要注意，sqoop有两个版本：

sqoop-1.4.4.bin__hadoop-1.0.0.tar.gz（对应hadoop1版本）

sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz（对应hadoop2版本）

出现上面的错误就是hadoop和对应的sqoop版本不一致，二者保持一致即可解决问题。