unbuntu14.04配置hadoop2.6.0的过程

一、需要软件

ubuntu14.04LTS

hadoop2.6.0

jdk

eclipse

二、环境配置

2.1修改bashrc文件

sudo gedit ~/.bashrc

java环境配置，添加以下内容

export JAVA_HOME=/home/dida/jdk1.7.0_75   

export JRE_HOME=${JAVA_HOME}/jre  

export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib  

export PATH=${JAVA_HOME}/bin:$PATH

hadoop配置，添加以下内容

#HADOOP VARIABLES START

export JAVA_HOME=/home/dida/jdk1.7.0_75

export HADOOP_INSTALL=/home/dida/hadoop

export PATH=$PATH:$HADOOP_INSTALL/bin

export PATH=$PATH:$HADOOP_INSTALL/sbin

export HADOOP_MAPRED_HOME=$HADOOP_INSTALL

export HADOOP_COMMON_HOME=$HADOOP_INSTALL

export HADOOP_HDFS_HOME=$HADOOP_INSTALL

export YARN_HOME=$HADOOP_INSTALL

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native

export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"

#HADOOP VARIABLES END

2.2修改hadoop-evn.sh

sudo gedit ~/hadoop/etc/hadoop/hadoop-env.sh

修改 JAVA_HOME=/home/dida/jdk1.7.0_75

2.3配置SSH免密码登录

下载ssh

sudo apt-get install ssh

安装完成后，启动服务

sudo /etc/init.d/ssh start

查看服务是否正确启动

ps -e | grep ssh

ssh生成私匙和公匙

ssh-keygen -t rsa -P ""

此时会在/home/dida/.ssh下生成两个文件：id_rsa和id_rsa.pub，前者为私钥，后者为公钥。
下面我们将公钥追加到authorized_keys中，它用户保存所有允许以当前用户身份登录到ssh客户端用户的公钥内容。
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys

登录ssh
ssh localhost

退出 exit

三、WordCount 测试

进在/home/dida/hadoop/路径下创建input文件夹

mkdir input

拷贝README.txt到input

cp README.txt input

执行WordCount

bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.6.0-sources.jar org.apache.hadoop.examples.WordCount input output

最后通过以下命令查看结果：

执行cat output/*，查看字符统计结果