Using sqoop to move data from mysql to hive

Sqoop only works with CDH3, so we have to install CDH3.

To install CDH3 on a Red Hat system:

1. Add the yum repository of your choice. Download this cloudera-cdh3.repo file for your RHEL/CentOS 5 system, or download this other cloudera-cdh3.repo file for your RHEL 6 system, and save it in your /etc/yum.repos.d/ directory.

   
   Note
   To install a different version of CDH on a Red Hat system, open the cloudera-cdh3.repo file and change the 3 in the repo file to the version number you want. For example, change the 3 to 3b4 to install CDH3 Beta 4.

2. Update the yum package index by running:

   $ sudo yum update yum
   

3. Find and install the Hadoop core package. For example:

$ yum search hadoop
$ sudo yum install hadoop-0.20

Note
In prior versions of CDH, the hadoop-0.20 package contained all of the service scripts in /etc/init.d/. In CDH3, the hadoop-0.20 package does not contain any service scripts – instead, those scripts are contained in the hadoop-0.20-<daemon> packages. Do the following step to install the daemon packages.

4. Install each type of daemon package on the appropriate machine. For example, install the NameNode package on your NameNode machine:

$ sudo yum install hadoop-0.20-<daemon type>

where <daemon type> is one of the following:

namenode

datanode

secondary namenode

jobtracker

tasktracker

5. Install the appropriate CDH component(s) that you want to use. SeeInstalling CDH3 Components

On Red Hat systems, type:

$ sudo yum install <CDH3-component-name>

To install this CDH3 component
Use this CDH3 component name

Flume
flume

Sqoop
sqoop

Hue
hue

Pig
hadoop-pig

Hive
hadoop-hive

HBase
hadoop-hbase

ZooKeeper
hadoop-zookeeper

Oozie server
oozie

Oozie client
oozie-client

Whirr
whirr

Then copy mysql-connector-java-5.1.10.jar to sqoop dir