以下操作都在普通用户下进行
1、解压 Sqoop
使用下面的命令,解压 Sqoop 安装包:
tar -zxvf sqoop-1.4.5.bin__hadoop-2.0.4-alpha.tar.gz
mv sqoop-1.4.5.bin__hadoop-2.0.4-alpha sqoop
cd sqoop
2、配置 Sqoop
2.1 配置 MySQL
mysql-connector-java-5.1.27-bin.jar
在mysql-connector-java-5.1.27-bin.jar所在目径下进行操作:
cp mysql-connector-java-5.1.27-bin.jar ~/sqoop/lib/
~表示家目录,/home/zkpk/
2.2 配置环境变量
cd ~/sqoop/conf
cp sqoop-env-template.sh sqoop-env.sh
将该文件 sqoop-env.sh 内容替换为:
#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/home/zkpk/hadoop
#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/home/zkpk/hadoop
//目前还没有对HBase进行安装,此处可以不用设置
#set the path to where bin/hbase is available
#export HBASE_HOME=
#Set the path to where bin/hive is available
export HIVE_HOME=/home/zkpk/hive
//目前还没有对Zookeeper进行安装,此处可以不用设置
#Set the path for where zookeper config dir is
#export ZOOCFGDIR=
3、启动并验证 Sqoop
进入 Sqoop 安装主目录:
cd ~/sqoop-1.4.5.bin__hadoop-2.0.4-alpha
bin/sqoop help
将数据从HDFS导出到MySql中:
在sqoop/bin目录下操作:注意空格
./sqoop export --connect jdbc:mysql://master:3306/test --username hadoop --password hadoop --table stus --export-dir '/in/stu' --fields-terminated-by '\t' -m 1
./sqoop export
--connect jdbc:mysql://master:3306/test //导出的数据库
--username hadoop //登录MySql的用户名
--password hadoop //登录MySql的密码
--table stus //MySql数据库中的表
--export-dir '/in/stu' //从HDFS中导出的路径
--fields-terminated-by '\t' -m 1 //分隔符为"\t",map(reduce)1个
mysql> alter database test character set utf8;
vi /etc/my.cnf,修改以下内容:
[mysqld]
default-character-set=utf8
character_set_server=utf8
init_connect='SET NAMES utf8'
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
[mysql]
default-character-set=utf8
[client]
default-character-set=utf8
1、解压 Sqoop
使用下面的命令,解压 Sqoop 安装包:
tar -zxvf sqoop-1.4.5.bin__hadoop-2.0.4-alpha.tar.gz
mv sqoop-1.4.5.bin__hadoop-2.0.4-alpha sqoop
cd sqoop
2、配置 Sqoop
2.1 配置 MySQL
mysql-connector-java-5.1.27-bin.jar
在mysql-connector-java-5.1.27-bin.jar所在目径下进行操作:
cp mysql-connector-java-5.1.27-bin.jar ~/sqoop/lib/
~表示家目录,/home/zkpk/
2.2 配置环境变量
cd ~/sqoop/conf
cp sqoop-env-template.sh sqoop-env.sh
将该文件 sqoop-env.sh 内容替换为:
#Set path to where bin/hadoop is available
export HADOOP_COMMON_HOME=/home/zkpk/hadoop
#Set path to where hadoop-*-core.jar is available
export HADOOP_MAPRED_HOME=/home/zkpk/hadoop
//目前还没有对HBase进行安装,此处可以不用设置
#set the path to where bin/hbase is available
#export HBASE_HOME=
#Set the path to where bin/hive is available
export HIVE_HOME=/home/zkpk/hive
//目前还没有对Zookeeper进行安装,此处可以不用设置
#Set the path for where zookeper config dir is
#export ZOOCFGDIR=
3、启动并验证 Sqoop
进入 Sqoop 安装主目录:
cd ~/sqoop-1.4.5.bin__hadoop-2.0.4-alpha
bin/sqoop help
将数据从HDFS导出到MySql中:
在sqoop/bin目录下操作:注意空格
./sqoop export --connect jdbc:mysql://master:3306/test --username hadoop --password hadoop --table stus --export-dir '/in/stu' --fields-terminated-by '\t' -m 1
./sqoop export
--connect jdbc:mysql://master:3306/test //导出的数据库
--username hadoop //登录MySql的用户名
--password hadoop //登录MySql的密码
--table stus //MySql数据库中的表
--export-dir '/in/stu' //从HDFS中导出的路径
--fields-terminated-by '\t' -m 1 //分隔符为"\t",map(reduce)1个
mysql> alter database test character set utf8;
vi /etc/my.cnf,修改以下内容:
[mysqld]
default-character-set=utf8
character_set_server=utf8
init_connect='SET NAMES utf8'
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
[mysql]
default-character-set=utf8
[client]
default-character-set=utf8