文章目录
1. 总结
在查找CDH 中hive的日志 看到一篇很不错的文章, 总结了 集群配置以及日志等安装目录的位置 这里转载过来方便自己查看
转载出处 : CDH集群配置、日志、jar包以及安装目录汇总
2. 关键目录
2.1 开启应用目录:默认可以直接敲命令行
/opt/cloudera/parcels/CDH/bin
查询
# ls
avro-tools kite-dataset sqoop-create-hive-table
beeline kudu sqoop-eval
bigtop-detect-javahome llama sqoop-export
catalogd llamaadmin sqoop-help
cli_mt load_gen sqoop-import
cli_st mahout sqoop-import-all-tables
flume-ng mapred sqoop-job
hadoop oozie sqoop-list-databases
hadoop-0.20 oozie-setup sqoop-list-tables
hadoop-fuse-dfs parquet-tools sqoop-merge
hadoop-fuse-dfs.orig pig sqoop-metastore
hbase pyspark sqoop-version
hbase-indexer sentry statestored
hbase-indexer-sentry solrctl whirr
hcat spark-shell yarn
hdfs spark-submit zookeeper-client
hive sqoop zookeeper-server
hiveserver2 sqoop2 zookeeper-server-cleanup
impala-collect-minidumps sqoop2-server zookeeper-server-initialize
impalad sqoop2-tool
impala-shell sqoop-codegen
2.2 zookeeper客户端命令位置
/opt/cloudera/parcels/CDH/lib/zookeeper/bin/zkCli.sh
2.3 日志目录
/var/log/下面有对应节点所开启服务的日志
其中
cloudera-scm-agent CDH客户端日志
cloudera-scm-server CDH服务端日志
hadoop-hdfs hadoop日志目录
hadoop-mapreduce
hadoop-yarn
hbase hbase日志目录
hive hive日志目录
zookeeper zookeeper日志目录
2.4 solr日志位置
tail -f /var/log/solr/solr-cmf-solr-SOLR_SERVER-...(hostname).log.out
2.5 配置文件目录
(1) 各个部件配置
/etc/cloudera-scm-agent/config.ini cm agent的配置目录
/etc/cloudera-scm-server/ cm server的配置目录
/etc/hadoop/conf Hadoop各个组件的配置
/etc/hive/conf hive配置文件目录
/etc/hbase/conf hbase配置文件目录
/opt/cloudera/parcels/CDH/etc是hadoop集群以及组件的配置文件文件夹
(2) 服务运行时所有组件的配置文件目录
/var/run/cloudera-scm-agent/process/
查看内容
# ls
244-hdfs-NAMENODE
245-hdfs-SECONDARYNAMENODE
246-hdfs-NAMENODE-nnRpcWait
247-zookeeper-server
250-yarn-RESOURCEMANAGER
254-yarn-JOBHISTORY
255-hive-HIVESERVER2
256-hive-HIVEMETASTORE
260-hbase-MASTER
280-collect-host-statistics
283-host-inspector
291-cluster-host-inspector
296-cluster-host-inspector
302-hdfs-NAMENODE
303-hdfs-SECONDARYNAMENODE
304-hdfs-NAMENODE-nnRpcWait
305-hive-HIVEMETASTORE
308-collect-host-statistics
311-host-inspector
320-collect-host-statistics
323-host-inspector
333-collect-host-statistics
336-host-inspector
343-yarn-RESOURCEMANAGER
347-yarn-JOBHISTORY
348-hive-HIVESERVER2
349-hive-HIVEMETASTORE
359-yarn-RESOURCEMANAGER
363-yarn-JOBHISTORY
364-hive-HIVESERVER2
365-hive-HIVEMETASTORE
377-collect-host-statistics
380-host-inspector
391-collect-host-statistics
394-host-inspector
403-collect-host-statistics
406-host-inspector
415-solr-SOLR_SERVER
ccdeploy_hadoop-conf_etchadoopconf.cloudera.hdfs_2394207936436282336
ccdeploy_hadoop-conf_etchadoopconf.cloudera.yarn_-5589955826917190240
ccdeploy_hadoop-conf_etchadoopconf.cloudera.yarn_-5844902762347635403
ccdeploy_hbase-conf_etchbaseconf.cloudera.hbase_1078272019303849382
ccdeploy_hive-conf_etchiveconf.cloudera.hive_-2148401581528977976
ccdeploy_hive-conf_etchiveconf.cloudera.hive_-9144129025529935636
ccdeploy_solr-conf_etcsolrconf.cloudera.solr_4855637747225843610
ccdeploy_spark2-conf_etcspark2conf.cloudera.spark2_on_yarn_-5389810983416787264
ccdeploy_spark2-conf_etcspark2conf.cloudera.spark2_on_yarn_-8416474417781205702
2.6 安装各个组件目录
/opt/cloudera/parcels/CDH
2.7 jar包目录
(1) 所有jar包所在目录
/opt/cloudera/parcels/CDH/jars
(2) 各个服务组件对应的jar包
/opt/cloudera/parcels/CDH/lib/
例如sqoop:/opt/cloudera/parcels/CDH/lib/sqoop/lib
2.8 Parcels包目录
(1) 服务软件包数据(parcels)
/opt/cloudera/parcel-repo/
(2) 服务软件包缓存数据
/opt/cloudera/parcel-cache/