环境准备
1)已经安装JDK1.7
2)已经搭建好Hadoop-2.2.0 开发环境
安装包下载
Sqoop安装包下载
下载地址:点击进入下载界面
进入下载界面可以看到上面的链接,点击
sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz进行下载保留到本地目录即可
相关jar下载
需要下载的jar包如下:
说明:jar包下载记得版本匹配,本次开发环境为hadoop-2.2.0,所以jar也应该是2.2.0版本的
commons-cli-1.2.jar
hadoop-common-2.2.0.jar
hadoop-mapreduce-client-core-2.2.0.jar
mysql-connector-java-5.1.21.jar
授人以鱼不如授人以渔
点击这里学习如何下载jar包
流程
1)执行以下命令切换至hadoop用户下: su hadoop
2)进入/usr/java目录:cd /usr/java
3)从本地上传sqoop安装包:rz,然后选择本地安装包
4)解压安装包:tar -zxvf sqoop-1.4.6.bin__hadoop-2.0.4-alpha.tar.gz
5)修改文件夹名称:mv sqoop-1.4.6.bin__hadoop-2.0.4-alpha sqoop
6)进入sqoop/conf目录:cd sqoop/conf
7)从模板复制sqoop-env.sh文件:cp sqoop-env-template.sh sqoop-env.sh
8)修改sqoop-env.sh文件:vi sqoop-env.sh
9)下面为修改内容:
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# included in all the hadoop scripts with source command
# should not be executable directly
# also should not be passed any arguments, since we need original $*
# Set Hadoop-specific environment variables here.
#Set path to where bin/hadoop is available
#export HADOOP_COMMON_HOME=/usr/java/hadoop
#Set path to where hadoop-*-core.jar is available
#export HADOOP_MAPRED_HOME=/usr/java/hadoop
#set the path to where bin/hbase is available
#export HBASE_HOME=
#Set the path to where bin/hive is available
#export HIVE_HOME=/usr/java/hive
#Set the path for where zookeper config dir is
#export ZOOCFGDIR=
~
~
~
~
"sqoop-env.sh" 35L, 1391C
修改的地方有:
export HADOOP_COMMON_HOME=/usr/java/hadoop
export HADOOP_MAPRED_HOME=/usr/java/hadoop
export HIVE_HOME=/usr/java/hive以上的黑体标识为修改的内容,HADOOP_COMMON_HOME等原本后面为空,填上你的安装目录即可
10)进入sqoop/lib目录并上传上面下载的jar包
11)切换至root用户
12)配置环境变量,修改 /etc/profile:vi /etc/profile
增加下面内容:
SQOOP_HOME=/usr/java/sqoop
PATH= $SQOOP_HOME/bin:$PATH
export SQOOP_HOME PATH
保存并退出
13)使环境变量生效:source /etc/profile
14)执行sqoop命令进行测试
sqoop list-databases --connect jdbc:mysql://localhost:3306/hive --username hive --password hive
list-databases:为显示数据库
–connect jdbc:mysql://localhost:3306/hive 为连接mysql
–username hive :用户名
–password hive:密码
其中的hive数据库和用户都是在上节课中创建的
下面为执行结果: