报错处理小记

1.hdfs使用put上传文件失败

防火墙没关

2.datanode起不来

多次格式化
每次namenode format会重新创建一个namenodeId,而data目录包含了上次format时的id,namenode format清空了namenode下的数据,但是没有清空datanode下的数据,导致启动时失败,所要做的就是每次fotmat前,清空data下的所有目录（即hdfs-site.xml文件中配置的dfs.data.dir目录）

3.没有jsp页面

上图是2.7
下面这个是2.6.5
好像2.7删除了jsp

4.mysql安装报错

直接删除原来的包

 yum -y remove mariadb-libs*

缺少net-tools
直接安装一个

yum install -y net-tools

5.vim用不了

[root@hadoop ~]# vim
vim: error while loading shared libraries: /lib64/libgpm.so.2: file too short

查找一下是哪里出错

yum provides *libgpm.so.2

Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirrors.aliyun.com
 * extras: mirrors.aliyun.com
 * updates: mirrors.aliyun.com
gpm-libs-1.20.7-6.el7.i686 : Dynamic library for for the gpm
Repo        : base
Matched from:
Provides    : libgpm.so.2
Filename    : /usr/lib/libgpm.so.2



gpm-libs-1.20.7-6.el7.x86_64 : Dynamic library for for the gpm
Repo        : base
Matched from:
Filename    : /usr/lib64/libgpm.so.2


gpm-libs-1.20.7-6.el7.x86_64 : Dynamic library for for the gpm
Repo        : @/
Matched from:
Filename    : /usr/lib64/libgpm.so.2

重新安装这个包

yum reinstall -y gpm-libs

就可以了

6.连接hive

which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/jdk/bin:/root/bin:/usr/jdk/bin:/usr/hadoop/bin:/usr/hadoop/sbin:/usr/jdk/bin:/usr/hadoop/bin:/usr/hadoop/sbin:/usr/hive/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/usr/hive/lib/hive-common-2.3.9.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.RuntimeException: java.net.ConnectException: Call From hadoop/192.168.26.148 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused

看看是不是hadoop没有起来

start-all.sh

就可以了

hadoop fs -chmod -R 777 /tmp

7.java连接hbase

今天在用java程序连接hbase出现错误

The package org.apache.hadoop.hbase is accessible from more than one module

错误的位置：是将其放在Modulepath目录下 ，也就是和jre放到了一起，这样是不对的。
正确的位置：应该放在Classpath下，这样就可以正常的运行了

连接超时

callTimeout=60000, callDuration=79694: can not resolve slave1,16020,1633736616091 row 'testTable,,' 

在

C:\Windows\System32\drivers\etc

hosts文件中添加IP映射

8.hdfs的web页面访问受限

报错如下

Permission denied: user=dr.who, access=READ_EXECUTE, inode="/tmp":root:supergroup:drwxrwx---

给权限

hdfs dfs -chmod -R 755 /tmp

9.jar包在linux运行缓慢

一个普通的java工程，。在eclipse下运行正常，但用eclipse达成jar包，在命令行执行却异常缓慢。

通过一些尝试，发现是因为打包的方式不对。eclipse导出可运行jar包有三种方式，分别为：

1.Extract required libraries into generated JAR

2.Package required libraries into generated JAR

3.Copy required libraries into a sub-folder next to the generated JAR

一开始选择了第二种，把需要的jar包全部打进了生成的JAR包中，这种情况下运行非常慢。之后尝试了第一种和第三种，运行正常了。

用解压软件查看生成的jar包，发现第一种是把引用的jar包中的类提取出来了。第三种，则是把引用的jar包放在了另一个文件夹中。这两种方式运行正常。

10.HADOOP3.xHDFS启动错误

HDFS格式化后启动dfs出现以下错误：

[root@master sbin]# ./start-dfs.sh
Starting namenodes on [master]
ERROR: Attempting to operate on hdfs namenode as root
ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting operation.
Starting datanodes
ERROR: Attempting to operate on hdfs datanode as root
ERROR: but there is no HDFS_DATANODE_USER defined. Aborting operation.
Starting secondary namenodes [slave1]
ERROR: Attempting to operate on hdfs secondarynamenode as root
ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting operation.

在/usr/hadoop/sbin路径下：
将start-dfs.sh，stop-dfs.sh两个文件顶部添加以下参数:

HDFS_DATANODE_USER=root
HDFS_DATANODE_SECURE_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root

还有，start-yarn.sh，stop-yarn.sh顶部也需添加以下：

YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root

11.MapReduce执行报错

 Job job_1634561573004_0001 failed with state FAILED due to: Application application_1634561573004_0001 failed 2 times due to Error launching appattempt_1634561573004_0001_000002. Got exception: org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.

YarnException: Unauthorized request to start container
各节点时间不同步，配置ntp即可
点击跳转ntp

12.Error: Cannot find configuration directory: /etc/hadoop

在hadoop-env.sh 配置一条hadoop配置文件所在目录

export HADOOP_CONF_DIR=/usr/hadoop/etc/hadoop/ 

刷新

source  hadoop-env.sh