![](https://img-blog.csdnimg.cn/20201014180756757.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
HDFS
源神
天道酬勤
展开
-
The variance for this alert is **MB which is 20% of the **MB average (**MB is the limit)
1、调整如下阀值2、检查HDFS文件系统使用率,清空HDFS上的.trash垃圾数据# su - hdfs # hdfs dfs -df -h# hadoop fs -expunge原创 2017-09-01 09:35:42 · 8197 阅读 · 0 评论 -
HDFS块block丢失导致Metrics Collector多次重启产生告警
因为集群升级扩容后多次断电、重启,可能导致存储在hbase中的监控数据的block丢了su hdfshadoop dfsadmin -safemode leave(退出安全模式)hadoop dfs -list-corruptfileblocks(检查HDFS丢失的block)hdfs fsck / -delete(个人发现丢失block都是跟监控相关的不重要数原创 2017-09-18 17:05:56 · 976 阅读 · 0 评论 -
java连接HDFS+Kerberos配置参数示例
必备参数如下:private void confLoad() throws IOException { conf = new Configuration(); conf.clear(); conf.set("hadoop.security.authentication", "kerberos"); conf.set("fs.defaultFS", "hdfs://172原创 2017-11-20 13:48:20 · 5162 阅读 · 0 评论 -
HDFS安装报错:Directory /data1/hadoop/hdfs/data is in an inconsistent state: cluster Id is incompatible
Ambari安装HDFS报错:2018-02-08 10:01:58,785 WARN common.Storage (DataStorage.java:loadDataStorage(449)) - Failed to add storage directory [DISK]file:/data2/hadoop/hdfs/data/org.apache.hadoop.hdfs.server.c...原创 2018-02-08 10:15:10 · 3486 阅读 · 0 评论 -
Hadoop常用操作HDFS命令
(1)查询:hadoop dfs -ls /(2)创建目录:hadoop dfs -mkdir -p /tmp/zy(3)本地文件拷贝至hdfs:hadoop dfs -put test.txt /tmp(4)查看文本内容:hadoop dfs -cat /tmp/test.txt(5)hdfs上文件下载至本地:hadoop dfs -get /tmp/test.txt /home/sgs...原创 2018-11-16 09:23:38 · 364 阅读 · 1 评论