1.hdfs使用put上传文件失败
防火墙没关
2.datanode起不来
多次格式化
每次namenode format会重新创建一个namenodeId,而data目录包含了上次format时的id,namenode format清空了namenode下的数据,但是没有清空datanode下的数据,导致启动时失败,所要做的就是每次fotmat前,清空data下的所有目录(即hdfs-site.xml文件中配置的dfs.data.dir目录)
3.没有jsp页面
上图是2.7
下面这个是2.6.5
好像2.7删除了jsp
4.mysql安装报错
直接删除原来的包
yum -y remove mariadb-libs*
缺少net-tools
直接安装一个
yum install -y net-tools
5.vim用不了
[root@hadoop ~]# vim
vim: error while loading shared libraries: /lib64/libgpm.so.2: file too short
查找一下是哪里出错
yum provides *libgpm.so.2
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.aliyun.com
* updates: mirrors.aliyun.com
gpm-libs-1.20.7-6.el7.i686 : Dynamic library for for the gpm
Repo : base
Matched from:
Provides : libgpm.so.2
Filename : /usr/lib/libgpm.so.2
gpm-libs-1.20.7-6.el7.x86_64 : Dynamic library for for the gpm
Repo : base
Matched from:
Filename : /usr/lib64/libgpm.so.2
gpm-libs-1.20.7-6.el7.x86_64 : Dynamic library for for the gpm
Repo : @/
Matched from:
Filename : /usr/lib64/libgpm.so.2
重新安装这个包
yum reinstall -y gpm-libs
就可以了
6.连接hive
which: no hbase in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/jdk/bin:/root/bin:/usr/jdk/bin:/usr/hadoop/bin:/usr/hadoop/sbin:/usr/jdk/bin:/usr/hadoop/bin:/usr/hadoop/sbin:/usr/hive/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in jar:file:/usr/hive/lib/hive-common-2.3.9.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.RuntimeException: java.net.ConnectException: Call From hadoop/192.168.26.148 to localhost:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused
看看是不是hadoop没有起来
start-all.sh
就可以了
hadoop fs -chmod -R 777 /tmp
7.java连接hbase
今天在用java程序连接hbase出现错误
The package org.apache.hadoop.hbase is accessible from more than one module
错误的位置:是将其放在Modulepath目录下 ,也就是和jre放到了一起,这样是不对的。
正确的位置:应该放在Classpath下,这样就可以正常的运行了
连接超时
callTimeout=60000, callDuration=79694: can not resolve slave1,16020,1633736616091 row 'testTable,,'
在
C:\Windows\System32\drivers\etc
hosts文件中添加IP映射
8.hdfs的web页面访问受限
报错如下
Permission denied: user=dr.who, access=READ_EXECUTE, inode="/tmp":root:supergroup:drwxrwx---
给权限
hdfs dfs -chmod -R 755 /tmp
9.jar包在linux运行缓慢
一个普通的java工程,。在eclipse下运行正常,但用eclipse达成jar包,在命令行执行却异常缓慢。
通过一些尝试,发现是因为打包的方式不对。eclipse导出可运行jar包有三种方式,分别为:
1.Extract required libraries into generated JAR
2.Package required libraries into generated JAR
3.Copy required libraries into a sub-folder next to the generated JAR
一开始选择了第二种,把需要的jar包全部打进了生成的JAR包中,这种情况下运行非常慢。之后尝试了第一种和第三种,运行正常了。
用解压软件查看生成的jar包,发现第一种是把引用的jar包中的类提取出来了。第三种,则是把引用的jar包放在了另一个文件夹中。这两种方式运行正常。
10.HADOOP3.xHDFS启动错误
HDFS格式化后启动dfs出现以下错误:
[root@master sbin]# ./start-dfs.sh
Starting namenodes on [master]
ERROR: Attempting to operate on hdfs namenode as root
ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting operation.
Starting datanodes
ERROR: Attempting to operate on hdfs datanode as root
ERROR: but there is no HDFS_DATANODE_USER defined. Aborting operation.
Starting secondary namenodes [slave1]
ERROR: Attempting to operate on hdfs secondarynamenode as root
ERROR: but there is no HDFS_SECONDARYNAMENODE_USER defined. Aborting operation.
在/usr/hadoop/sbin路径下:
将start-dfs.sh,stop-dfs.sh两个文件顶部添加以下参数:
HDFS_DATANODE_USER=root
HDFS_DATANODE_SECURE_USER=hdfs
HDFS_NAMENODE_USER=root
HDFS_SECONDARYNAMENODE_USER=root
还有,start-yarn.sh,stop-yarn.sh顶部也需添加以下:
YARN_RESOURCEMANAGER_USER=root
HADOOP_SECURE_DN_USER=yarn
YARN_NODEMANAGER_USER=root
11.MapReduce执行报错
Job job_1634561573004_0001 failed with state FAILED due to: Application application_1634561573004_0001 failed 2 times due to Error launching appattempt_1634561573004_0001_000002. Got exception: org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container.
YarnException: Unauthorized request to start container
各节点时间不同步,配置ntp即可
点击跳转ntp
12.Error: Cannot find configuration directory: /etc/hadoop
在hadoop-env.sh 配置一条hadoop配置文件所在目录
export HADOOP_CONF_DIR=/usr/hadoop/etc/hadoop/
刷新
source hadoop-env.sh