1. yarn和hdfs启动停止问题
dfs:
HDFS_DATANODE_USER=root
HADOOP_SECURE_DN_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root
yarn:
RN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root
YARN_RESOURCEMANAGER_USER=root
2.mapreduce启动不起来的问题:运行一直卡在 job的编号地方
yarn-site.xml 配置这个:
value值的获取: hadoop classpath
<property>
<name>yarn.application.classpath</name>
<value>/opt/module/hadoop-3.1.3/etc/hadoop:/opt/module/hadoop-3.1.3/share/hadoop/common/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/common/*:/opt/module/hadoop-3.1.3/share/hadoop/hdfs:/opt/module/hadoop-3.1.3/share/hadoop/hdfs/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/hdfs/*:/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/mapreduce/*:/opt/module/hadoop-3.1.3/share/hadoop/yarn:/opt/module/hadoop-3.1.3/share/hadoop/yarn/lib/*:/opt/module/hadoop-3.1.3/share/hadoop/yarn/*</value>
</property>
3.mapreduce让检查配置问题:Please check whether your etc/hadoop/mapred-site.xml contains the below configuration:
4. 解决内存资源不足的问题
yarn-site.xml
<!--Start--===========解决内存不足问题===============-->
<property>
<name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name>
<value>95.0</value>
</property>
<!--集群内存-->
<property>
<name>yarn.nodemanager.resource.memory-mb</name>
<value>2048</value>
</property>
<!--调度程序最小值-分配-->
<property>
<name>yarn.scheduler.minimum-allocation-mb</name>
<value>2048</value>
</property>
<!--比率,具体是啥比率还没查...-->
<property>
<name>yarn.nodemanager.vmem-pmem-ratio</name>
<value>2.1</value>
</property>
<!--End--解决内存不足问题-->
5. namenode启动, 格式化失败问题:
原因: 未删除原来的id,导致新生成的集群id和原id不对应, namenode启动失败
解决:
1.如果格式化失败就将namenode存放数据在所有集群的地址进行删除,重新再进行namenode
2. 或者将data中和name中的clusterID修改为一致
6. Root不能模拟Root用户的问题: User: root is not allowed to impersonate root (state=,code=0)
解决: core-site.xml添加
<!--core-site.xml里加上下面的配置文件,重启hdfs,其中“xxx”是连接beeline的用户,将“xxx”替换成自己的用户名root即可-->
<property>
<name>hadoop.proxyuser.xxx.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.xxx.groups</name>
<value>*</value>
</property>
7.网页上不能操作文件的问题
<!--指定页面静态用户-->
<property>
<name>hadoop.http.staticuser.user</name>
<value>root</value>
</property>