hdfs
stone_fall
程序员+宅男
展开
-
hadoop | 服务器分布式安装hadoop,hbase,zookeeper,phoenix,spark | 集群启动顺序 | hbase命令
文章目录系统与版本集群结构安装系统与版本在vSphere ESXI服务器中安装大数据集群,系统镜像为CentOS-7-x86_64-Minimal-1908.iso版本CentOS-7-x86_64-Minimal-1908.isojdk-8u211-linux-x64apache-phoenix-5.0.0-HBase-2.0-binapache-zookeeper-3.5.5-binhadoop-3.1.2hbase-2.0.5spark-2.原创 2020-09-14 14:36:10 · 1458 阅读 · 0 评论 -
Centos 7 | zookeeper | Error contacting service. It is probably not running | 解决方案
文章目录问题描述解决方案问题描述各个节点zkServer.sh start启动后,查看状态zkServer.sh status出现报错Error contacting service. It is probably not running查看zookeeper日志,有类似警告Cannot open channel to * at election address *java.net.ConnectException: Connection refused解决方案网上有很多解决方案检查原创 2020-09-14 08:03:40 · 452 阅读 · 0 评论 -
HBase 数据备份 | export | import
文章目录导出export导入import导出export以test表为例删除/移动原有备份hadoop fs -rm -r /opt/hbase-data-back/test_back创建表hbase shellcreate 'test','info'插入数据put 'test','01','info:msg','hello'put 'test','02','info:msg','hello'put 'test','03','info:msg','hello'导原创 2020-08-10 15:20:18 · 473 阅读 · 0 评论 -
Hadoop启动顺序 | jps对应进程 | hadoop | zookeeper | hbase | spark
Hadoop启动hadoop集群后,可以通过jps命令显示当前所有java进程pid的命令,也就显示了所有与Hadoop相关的进程。启动hadoopstart-all.sh主节点5664 SecondaryNameNode20898 Jps5821 ResourceManager5439 NameNode从节点21557 NodeManager21432 DataNode24765 Jps启动zookeeperzkServer.sh start启动节点新增进程原创 2020-06-29 16:17:01 · 2760 阅读 · 0 评论 -
Flume实例一 | 监控服务器Lynis日志上传hdfs | 监控目录上传hdfs
由于lynis的日志是直接覆盖原来的文件,似乎无法通过exec等source对其进行监控(我写的agent都shut down了)。如果有同学成功了并且愿意分享的话,我不甚感激。所以我的尝试以下:思路为,通过shell监控lynis日志目录,如果有新日志以**时间命名的方式**就存储在目标目录`/var/log/lynis/cp_logs`中,再通过`Spooling Directory Source`的`flume agent`对目录进行监控。原创 2020-05-30 08:25:44 · 840 阅读 · 0 评论 -
Centos 7 | Flume 安装与初步使用 | telnet
环境产品版本Centos 73.10.0-1062.9.1.el7.x86_64flumeapache-flume-1.9.0-bin安装与配置下载地址清华镜像https://mirrors.tuna.tsinghua.edu.cn/apache/flume/解压tar -zxvf apache-flume-1.9.0-bin.tar.gz创建软链接ln -s apache-flume-1.8.0-bin/ flume配置环境变量vi ~/.b原创 2020-05-18 15:38:49 · 676 阅读 · 0 评论 -
hdfs | 开启hbase后hadoop崩溃
文章目录错误一览解决方案错误一览开启hbase后hadoop崩溃。重启hadoop。start-all.sh启动hadoop顺利(单节点)[hadoop@master logs]$ start-all.shWARNING: Attempting to start all Apache Hadoop daemons as hadoop in 10 seconds.WARNING: This is not a recommended production deployment configura原创 2020-05-24 21:28:36 · 785 阅读 · 0 评论 -
hadoop | clusterid不一致 | Unable to load native-hadoop library for your platform
文章目录错误一览解决方案错误一览执行命令sqlline.py master:2181出错20/05/24 10:11:05 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable20/05/24 10:11:39 WARN client.ConnectionImplementation: Retri原创 2020-05-24 11:11:42 · 1215 阅读 · 0 评论 -
hadoop | start-yarn.sh与stop-yarn.sh执行失败
文章目录错误一览解决方案错误一览sbin/start-yarn.sh 执行失败Starting resourcemanagerERROR: Attempting to operate on yarn resourcemanager as rootERROR: but there is no YARN_RESOURCEMANAGER_USER defined. Aborting operation.Starting nodemanagersERROR: Attempting to opera原创 2020-05-24 10:04:06 · 3072 阅读 · 0 评论 -
hadoop的启动和停止命令
sbin/start-all.sh 启动所有的Hadoop守护进程。包括NameNode、 Secondary NameNode、DataNode、ResourceManager、NodeManagersbin/stop-all.sh 停止所有的Hadoop守护进程。包括NameNode、 Secondary NameNode、DataNode、ResourceManager、NodeManagersbin/start-dfs.sh 启动Hadoop HDFS守护进程NameNode、Seconda转载 2020-05-24 09:17:31 · 1294 阅读 · 0 评论