hadoop2.6.3安装
Ubuntu 14.04 LTS 64位版本上安装hadoop 2.6.3过程记录
- 安装JDK
- 安装并设置ssh免密码登录
- 安装hadoop2.6.3
安装jdk并设置好JDK环境变量
jdk的具体安装和设置这里不赘述,网上一大堆参考文章。
安装并设置ssh免密码登录
ssh不是hadoop的依赖环境,但是如果你要登录管理成百上千台hadoop主机,ssh的免密码登录必不可少。 shell命令如下
sudo apt-get update
sudo apt-get install ssh
ssh-keygen -t dsa -P '' -f ~/.ssh/id_rsa
cat .ssh/id_rsa.pub >> ~/.ssh/authorized_keys
执行 ssh localhost
如果显示如下信息,说明ssh免密码登录已经配置好了,第一次登录时会询问是否继续,输入yes即可。
The authenticity of host 'localhost (127.0.0.1)' can't be established.
ECDSA key fingerprint is ec:67:f0:9f:51:ac:5b:2d:6a:d3:61:3c:2e:93:1d:81.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.13.0-65-generic x86_64)
* Documentation: https://help.ubuntu.com/
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
安装hadoop-2.6.3
下载并解压hadoop-2.6.3.tar.gz
sudo mkdir -p /opt/hadoop
cd /opt/hadoop
wget http://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-2.6.3/hadoop-2.6.3.tar.gz
tar -zxvf hadoop-2.6.3.tar.gz
配置以下3个配置文件
etc/hadoop/core-site.xml
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>
</configuration>
etc/hadoop/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
etc/hadoop/mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>localhost:9001</value>
</property>
</configuration>
格式化hadoop的文件系统HDFS
./bin/hdfs namenode -format
启动hadoop所有进程
./sbin/start-dfs.sh
./sbin/start-yarn.sh
提示 localhost: Error: JAVA_HOME is not set and could not be found.
修改etc/hadoop/hadoop-env.sh文件中下面这一行:
export JAVA_HOME=${JAVA_HOME}
修改为使用绝对路径值
export JAVA_HOME=/linuxdata/apps/jdk1.8.0_45 # 绝对路径
重新执行上述启动命令,即可启动,提示如下:
Starting namenodes on [localhost]
localhost: starting namenode, logging to /linuxdata/apps/hadoop-2.6.3/logs/hadoop-root-namenode-kline-r400.out
localhost: starting datanode, logging to /linuxdata/apps/hadoop-2.6.3/logs/hadoop-root-datanode-kline-r400.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /linuxdata/apps/hadoop-2.6.3/logs/hadoop-root-secondarynamenode-kline-r400.out
starting yarn daemons
starting resourcemanager, logging to /linuxdata/apps/hadoop-2.6.3/logs/yarn-root-resourcemanager-kline-r400.out
localhost: starting nodemanager, logging to /linuxdata/apps/hadoop-2.6.3/logs/yarn-root-nodemanager-kline-r400.out
到此,你的hadoop就已经安装并成功启动。