伪分布式部署Hadoop

#### install java

$ wget http://mirrors.tuna.tsinghua.edu.cn/apache/hadoop/common/hadoop-3.1.2/hadoop-3.1.2.tar.gz
$ tar xzf hadoop-3.1.2.tar.gz
$ gedit ~/.bashrc
export HADOOP_HOME=/mnt/hgfs/hadoop-env/hadoop-3.1.2
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
$ source ~/.bashrc
$ hadoop version
Hadoop 3.1.2
Source code repository https://github.com/apache/hadoop.git -r 1019dde65bcf12e05ef48ac71e84550d589e5d9a
Compiled by sunilg on 2019-01-29T01:39Z
Compiled with protoc 2.5.0
From source with checksum 64b8bdd4ca6e77cce75a93eb09ab2a9
This command was run using /mnt/hgfs/hadoop-env/hadoop-3.1.2/share/hadoop/common/hadoop-common-3.1.2.jar

$ cd $HADOOP_HOME
$ mkdir my_config
$ gedit ~/.bashrc
export HADOOP_CONF_DIR=/mnt/hgfs/hadoop-env/hadoop-3.1.2/my_config
$ source ~/.bashrc
$ cd my_config
$ cp -r ../etc/hadoop/* .
$ gedit core-site.xml
$ gedit hdfs-site.xml
$ gedit mapred-site.xml
$ gedit yarn-site.xml

$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
$ ssh-add
$ sudo netstat -ntlp|grep 22
####if hasn't start
####$ service ssh start
####if not found
####$ sudo apt-get install openssh-server
####$ sudo netstat -ntlp|grep 22
$ ssh localhost

$ hdfs namenode -format
$ start-dfs.sh
####if localhost: ERROR: JAVA_HOME is not set and could not be found.
####在$HADOOP_CONF_DIR/hadoop-env.sh文件头部再显示地重新声明一遍JAVA_HOME
$ start-yarn.sh
$ mr-jobhistory-daemon.sh start historyserver

$ jps
$ netstat -nap | grep 4759
####http://localhost:9870 (namenode)
####http://locahhost:8088  (resource manager)
####http://localhost:19888 (history)

$ mr-jobhistory-daemon.sh stop historyserver
$ stop-yarn.sh
$ stop-dfs.sh

 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值