CDH集群配置、日志、jar包以及安装目录和常用命令汇总

一、总结目的

因为CDH集群的配置、日志、jar包以及安装文件的目录非常多,而且分散,每次绕进去就有种绕不出来的感觉。所以,下决心总结一下,防止自己后面采坑。

二、关键目录

1.开启应用目录:默认可以直接敲命令行

/opt/cloudera/parcels/CDH/bin
查询
# ls
avro-tools                kite-dataset   sqoop-create-hive-table
beeline                   kudu           sqoop-eval
bigtop-detect-javahome    llama          sqoop-export
catalogd                  llamaadmin     sqoop-help
cli_mt                    load_gen       sqoop-import
cli_st                    mahout         sqoop-import-all-tables
flume-ng                  mapred         sqoop-job
hadoop                    oozie          sqoop-list-databases
hadoop-0.20               oozie-setup    sqoop-list-tables
hadoop-fuse-dfs           parquet-tools  sqoop-merge
hadoop-fuse-dfs.orig      pig            sqoop-metastore
hbase                     pyspark        sqoop-version
hbase-indexer             sentry         statestored
hbase-indexer-sentry      solrctl        whirr
hcat                      spark-shell    yarn
hdfs                      spark-submit   zookeeper-client
hive                      sqoop          zookeeper-server
hiveserver2               sqoop2         zookeeper-server-cleanup
impala-collect-minidumps  sqoop2-server  zookeeper-server-initialize
impalad                   sqoop2-tool
impala-shell              sqoop-codegen

(1)zookeeper客户端命令位置

/opt/cloudera/parcels/CDH/lib/zookeeper/bin/zkCli.sh

2.日志目录(非常关键)

/var/log/下面有对应节点所开启服务的日志
其中
cloudera-scm-agent  CDH客户端日志
cloudera-scm-server CDH服务端日志
hadoop-hdfs hadoop日志目录
hadoop-mapreduce
hadoop-yarn
hbase hbase日志目录
hive hive日志目录
zookeeper  zookeeper日志目录

(1)solr日志位置

tail -f /var/log/solr/solr-cmf-solr-SOLR_SERVER-...(hostname).log.out

3.配置文件目录(非常关键)

(1)各个部件配置

/etc/cloudera-scm-agent/config.ini cm agent的配置目录
/etc/cloudera-scm-server/ cm server的配置目录

/etc/hadoop/conf Hadoop各个组件的配置
/etc/hive/conf  hive配置文件目录
/etc/hbase/conf  hbase配置文件目录

/opt/cloudera/parcels/CDH/etc是hadoop集群以及组件的配置文件文件夹 

(2)服务运行时所有组件的配置文件目录

/var/run/cloudera-scm-agent/process/
查看内容
# ls
244-hdfs-NAMENODE
245-hdfs-SECONDARYNAMENODE
246-hdfs-NAMENODE-nnRpcWait
247-zookeeper-server
250-yarn-RESOURCEMANAGER
254-yarn-JOBHISTORY
255-hive-HIVESERVER2
256-hive-HIVEMETASTORE
260-hbase-MASTER
280-collect-host-statistics
283-host-inspector
291-cluster-host-inspector
296-cluster-host-inspector
302-hdfs-NAMENODE
303-hdfs-SECONDARYNAMENODE
304-hdfs-NAMENODE-nnRpcWait
305-hive-HIVEMETASTORE
308-collect-host-statistics
311-host-inspector
320-collect-host-statistics
323-host-inspector
333-collect-host-statistics
336-host-inspector
343-yarn-RESOURCEMANAGER
347-yarn-JOBHISTORY
348-hive-HIVESERVER2
349-hive-HIVEMETASTORE
359-yarn-RESOURCEMANAGER
363-yarn-JOBHISTORY
364-hive-HIVESERVER2
365-hive-HIVEMETASTORE
377-collect-host-statistics
380-host-inspector
391-collect-host-statistics
394-host-inspector
403-collect-host-statistics
406-host-inspector
415-solr-SOLR_SERVER
ccdeploy_hadoop-conf_etchadoopconf.cloudera.hdfs_2394207936436282336
ccdeploy_hadoop-conf_etchadoopconf.cloudera.yarn_-5589955826917190240
ccdeploy_hadoop-conf_etchadoopconf.cloudera.yarn_-5844902762347635403
ccdeploy_hbase-conf_etchbaseconf.cloudera.hbase_1078272019303849382
ccdeploy_hive-conf_etchiveconf.cloudera.hive_-2148401581528977976
ccdeploy_hive-conf_etchiveconf.cloudera.hive_-9144129025529935636
ccdeploy_solr-conf_etcsolrconf.cloudera.solr_4855637747225843610
ccdeploy_spark2-conf_etcspark2conf.cloudera.spark2_on_yarn_-5389810983416787264
ccdeploy_spark2-conf_etcspark2conf.cloudera.spark2_on_yarn_-8416474417781205702

4.安装各个组件目录

/opt/cloudera/parcels/CDH

5.jar包目录

(1)所有jar包所在目录

/opt/cloudera/parcels/CDH/jars

(2)各个服务组件对应的jar包

/opt/cloudera/parcels/CDH/lib/

例如sqoop:/opt/cloudera/parcels/CDH/lib/sqoop/lib

6.Parcels包目录

(1)服务软件包数据(parcels)

/opt/cloudera/parcel-repo/

(2)服务软件包缓存数据

/opt/cloudera/parcel-cache/

(舒了一口气~)

三、常用命令

1.操作CM

(1)启动cm

systemctl start cloudera-scm-server

(2)重启cm

systemctl restart cloudera-scm-server

(3)停止

systemctl stop cloudera-scm-server

2.查看日志

tail -f /var/log/kafka/kafka-broker-cdh1.log
tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log

3.查看组件版本号

(1)选择任意主机:选择Hosts->All Hosts->选择任意一个主机

(2)选择组件:Components即可看到

(又舒了一口气~)

  • 7
    点赞
  • 28
    收藏
    觉得还不错? 一键收藏
  • 4
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 4
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值