修改主机名称
vim /etc/hostname
写上想要的名字即可
===========================================
修改IP和主机名映射,
本来需要通过IP+端口进行访问,配置映射后直接通过 映射名+端口 即可
vim /etc/hosts
写上想要映射的IP和映射名,比如想通过 hadoop1:8080访问192.168.136.103:8080
在/etc/hosts中写上如下一行即可
192.168.136.103 hadoop1
===========================================
开启SSH免密
任意文件夹下执行
ssh-keygen -t rsa
然后敲(三个回车),就会生成两个文件id_rsa(私钥)、id_rsa.pub(公钥)
将公钥拷贝到要免密登录的目标机器上
ssh-copy-id+目的地主机名
ssh-copy-id hadoop102
拷贝完成后,即可从目标主机上 使用 ssh+当前主机名 操控 当前主机
===========================================
ssh免密开启的前提下,快速分发文件的脚本
xsync
#!/bin/bash
#1. 判断参数个数
if [ $# -lt 1 ]
then
echo Not Enough Arguement!
exit;
fi
#2. 遍历集群所有机器
for host in hadoop104 hadoop105
do
echo ==================== $host ====================
#3. 遍历所有目录,挨个发送
for file in $@
do
#4. 判断文件是否存在
if [ -e $file ]
then
#5. 获取父目录
pdir=$(cd -P $(dirname $file); pwd)
#6. 获取当前文件的名称
fname=$(basename $file)
ssh $host "mkdir -p $pdir"
rsync -av $pdir/$fname $host:$pdir
else
echo $file does not exists!
fi
done
done
用法:xsync+要拷贝的文件夹路径
效果:会将指定的文件夹/文件 拷贝到目标主机的同一路径下,示例
xsync /opt/module/hadoop-3.1.3/etc/hadoop/
=============================
一键启停集群脚本
myhadoop.sh
#!/bin/bash
if [ $# -lt 1 ]
then
echo "No Args Input..."
exit ;
fi
case $1 in
"start")
echo " =================== 启动 hadoop集群 ==================="
echo " --------------- 启动 hdfs ---------------"
ssh hadoop103 "/opt/module/hadoop-3.1.3/sbin/start-dfs.sh"
echo " --------------- 启动 yarn ---------------"
ssh hadoop104 "/opt/module/hadoop-3.1.3/sbin/start-yarn.sh"
echo " --------------- 启动 historyserver ---------------"
ssh hadoop103 "/opt/module/hadoop-3.1.3/bin/mapred --daemon start historyserver"
;;
"stop")
echo " =================== 关闭 hadoop集群 ==================="
echo " --------------- 关闭 historyserver ---------------"
ssh hadoop103 "/opt/module/hadoop-3.1.3/bin/mapred --daemon stop historyserver"
echo " --------------- 关闭 yarn ---------------"
ssh hadoop104 "/opt/module/hadoop-3.1.3/sbin/stop-yarn.sh"
echo " --------------- 关闭 hdfs ---------------"
ssh hadoop105 "/opt/module/hadoop-3.1.3/sbin/stop-dfs.sh"
;;
*)
echo "Input Args Error..."
;;
esac
myhadoop.sh start
myhadoop.sh stop
======================
集群执行JPS脚本
jpsall
#!/bin/bash
for host in hadoop103 hadoop104 hadoop105
do
echo =============== $host ===============
ssh $host jps
done
jpsall
================
放在对应用户的/bin目录下即可