Hadoop伪分布式安装

草稿，未完待续

安装

修改配置

修改文件路径皆为 /opt/hadoop/hadoop-2.7.3/etc/hadoop

修改 hadoop-env.sh

cd /opt/hadoop/hadoop-2.7.3/etc/hadoop vi hadoop-env.sh export JAVA_HOME=/usr/lib/java/jdk1.7.0_79

修改 core-site.xml

<configuration>
 <property>
  <name>fs.defaultFS</name>
  <value>hdfs://localhost:9000</value>
 </property>
</configuration>

修改 hdfs-site.xml

含义只备份一个

<configuration>
 <property>
   <name>dfs.replication</name>
   <value>1</value>
 </property>

启动集群

格式化namenode

hdfs namenode -format

启动集群

./opt/hadoop/hadoop-2.7.3/sbin/start-dfs.sh

报错

Starting namenodes on [localhost] localhost: ssh: connect to host localhost port 22: Connection refused localhost: ssh: connect to host localhost port 22: Connection refused Starting secondary namenodes [0.0.0.0] 0.0.0.0: ssh: connect to host 0.0.0.0 port 22: Connection refused

启动服务，发现尚未安装

service sshd start Failed to start sshd.service: Unit sshd.service not found.

安装服务

sudo apt-get install openssh-server

查看服务是否安装

ps -e |grep ssh 5063 ? 00:00:00 sshd

启动服务

service sshd start

再次启动集群

start-dfs.sh

报错,看来得吧IP和主机名称设置一下

/opt/hadoop/hadoop-2.7.3/sbin$ ./start-dfs.sh Starting namenodes on [localhost] The authenticity of host 'localhost (127.0.0.1)' can't be established. ECDSA key fingerprint is SHA256:C2gR20UYgO3SAAvHtXo/bsGAczeFRz+TSk05aiu6fTA. Are you sure you want to continue connecting (yes/no)? y Please type 'yes' or 'no': yes localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts. austinye@localhost's password: localhost: starting namenode, logging to /opt/hadoop/hadoop-2.7.3/logs/hadoop-austinye-namenode-austinye-VirtualBox.out austinye@localhost's password: localhost: starting datanode, logging to /opt/hadoop/hadoop-2.7.3/logs/hadoop-austinye-datanode-austinye-VirtualBox.out Starting secondary namenodes [0.0.0.0] The authenticity of host '0.0.0.0 (0.0.0.0)' can't be established. ECDSA key fingerprint is SHA256:C2gR20UYgO3SAAvHtXo/bsGAczeFRz+TSk05aiu6fTA. Are you sure you want to continue connecting (yes/no)? yes 0.0.0.0: Warning: Permanently added '0.0.0.0' (ECDSA) to the list of known hosts. austinye@0.0.0.0's password: 0.0.0.0: starting secondarynamenode, logging to /opt/hadoop/hadoop-2.7.3/logs/hadoop-austinye-secondarynamenode-austinye-VirtualBox.out

转载于:https://my.oschina.net/u/1393778/blog/805388