Cloudera Manager : REST API

最新推荐文章于 2022-10-05 15:53:14 发布
最新推荐文章于 2022-10-05 15:53:14 发布
阅读量2.3k 收藏 5
点赞数 1
分类专栏： 大数据-cdh 文章标签： cdh监控api cd调用shell api cm rest api
版权声明：本文为博主原创文章，遵循 CC 4.0 BY 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/eyeofeagle/article/details/108843376
版权

文章目录

1, api使用指南

总览：http://test-c6:7180/static/apidocs/
quickstart: http://test-c6:7180/static/apidocs/tutorial.html

a, 查看集群信息

[root@test-c6 ~]# curl -u admin:admin 'http://localhost:7180/api/v17/clusters'
{
  "items" : [ {
    "name" : "Cluster 1",
    "displayName" : "Cluster 1",
    "version" : "CDH5",
    "fullVersion" : "5.12.0",
    "maintenanceMode" : false,
    "maintenanceOwners" : [ ],
    "clusterUrl" : "http://cdh-c6-master:7180/cmf/clusterRedirect/cluster_1",
    "hostsUrl" : "http://cdh-c6-master:7180/cmf/clusterRedirect/cluster_1/hosts",
    "entityStatus" : "UNKNOWN_HEALTH",
    "uuid" : "0e5f377c-2036-4cf2-a757-f39cacea8e6b"
  } ]
}

b, 查看主机信息

[root@test-c6 ~]# curl -u admin:admin  http://localhost:7180/api/v17/hosts
{
  "items" : [ {
    "hostId" : "68d905d8-4802-486d-9285-41e70576c05b",
    "ipAddress" : "192.168.56.161",
    "hostname" : "test-c6",
    "rackId" : "/default",
    "hostUrl" : "http://test-c6:7180/cmf/hostRedirect/68d905d8-4802-486d-9285-41e70576c05b",
    "maintenanceMode" : false,
    "maintenanceOwners" : [ ],
    "commissionState" : "COMMISSIONED",
    "numCores" : 1,
    "numPhysicalCores" : 1,
    "totalPhysMemBytes" : 5210898432
  }, {
    "hostId" : "0244fbb4-0d02-40b5-9cac-17d9e388eb97",
    "ipAddress" : "192.168.56.162",
    "hostname" : "test-c62",
    "rackId" : "/default",
    "hostUrl" : "http://test-c6:7180/cmf/hostRedirect/0244fbb4-0d02-40b5-9cac-17d9e388eb97",
    "maintenanceMode" : false,
    "maintenanceOwners" : [ ],
    "commissionState" : "COMMISSIONED",
    "numCores" : 1,
    "numPhysicalCores" : 1,
    "totalPhysMemBytes" : 2176856064
  } ]
}

c, 查看服务状态

1, 获取服务名称和状态

##查询所有服务
[root@test-c6 ~]# curl -u admin:admin  http://localhost:7180/api/v17/clusters/Cluster%201/services
{
  "items" : [ {
    "name" : "zookeeper",
    "type" : "ZOOKEEPER",
    "clusterRef" : {
      "clusterName" : "cluster"
    },
    "serviceUrl" : "http://test-c6:7180/cmf/serviceRedirect/zookeeper",
    "roleInstancesUrl" : "http://test-c6:7180/cmf/serviceRedirect/zookeeper/instances",
    "serviceState" : "STARTED",
    "healthSummary" : "BAD",
    "healthChecks" : [ {
      "name" : "ZOOKEEPER_CANARY_HEALTH",
      "summary" : "GOOD",
      "suppressed" : false
    }, {
      "name" : "ZOOKEEPER_SERVERS_HEALTHY",
      "summary" : "BAD",
      "suppressed" : false
    } ],
    "configStalenessStatus" : "FRESH",
    "clientConfigStalenessStatus" : "FRESH",
    "maintenanceMode" : false,
    "maintenanceOwners" : [ ],
    "displayName" : "ZooKeeper",
    "entityStatus" : "BAD_HEALTH"
  }, 
....
]}

## 查询单个服务
[root@test-c6 ~]# curl -u admin:admin  http://localhost:7180/api/v17/clusters/Cluster%201/services/hbase
{
  "name" : "hbase",
  "type" : "HBASE",
  "clusterRef" : {
    "clusterName" : "cluster"
  },
  "serviceUrl" : "http://test-c6:7180/cmf/serviceRedirect/hbase",
  "roleInstancesUrl" : "http://test-c6:7180/cmf/serviceRedirect/hbase/instances",
  "serviceState" : "STOPPED",
  "healthSummary" : "DISABLED",
  "healthChecks" : [ {
    "name" : "HBASE_MASTER_HEALTH",
    "summary" : "DISABLED",
    "suppressed" : false
  }, {
    "name" : "HBASE_REGION_SERVERS_HEALTHY",
    "summary" : "DISABLED",
    "suppressed" : false
  } ],
  "configStalenessStatus" : "FRESH",
  "clientConfigStalenessStatus" : "FRESH",
  "maintenanceMode" : false,
  "maintenanceOwners" : [ ],
  "displayName" : "HBase",
  "entityStatus" : "STOPPED"
}

2, 获取服务配置

在这里插入图片描述

[root@cdh-c6-master ~]# curl -u admin:admin  \
http://localhost:7180/api/v17/clusters/cluster_1/services/yarn?view=export
{
  "name" : "yarn",
  "type" : "YARN",
  "config" : {
    "items" : [  {
      "name" : "hadoop_secure_web_ui",
      "value" : "false",
      "sensitive" : false
    },
    .... ]
  },
  "roles" : [  ... ],
  "displayName" : "YARN (MR2 Included)",
  "roleConfigGroups" : [  {
    "name" : "yarn-JOBHISTORY-BASE",
    "displayName" : "JobHistory Server Default Group",
    "roleType" : "JOBHISTORY",
    "base" : true,
    "serviceRef" : {
      "clusterName" : "cluster_1",
      "serviceName" : "yarn"
    },
    "config" : {
      "items" : [ {
        "name" : "mr2_jobhistory_java_heapsize",
        "value" : "159383552",
        "sensitive" : false
      } ]
    }
  },  ... ]
}[root@cdh-c6-master ~]#

3, 修改服务配置

I, 修改cm监控配置 (调用rest接口)

在这里插入图片描述

[root@c7-iris2 ~]# cat a.json
{
  "items":
     [{
	   "name":"firehose_heapsize",
	   "value":1073741824
     }]
}

[root@c7-iris2 ~]# curl -X PUT -u admin:admin  \
http://localhost:7180/api/v17/cm/service/roleConfigGroups/mgmt-SERVICEMONITOR-BASE/config \
-H "Content-type: application/json" -d @a.json

##备注： -d @参数文件   --> 等价于直接填写参数： 
-d "{\"items\":[{\"name\":\"firehose_heapsize\",\"value\":1073741824}]}"

## 为方便提交命令，可以使用这种格式，如：
# 1073741824 bytes:1g
# 268435456  bytes:256m
curl -X PUT -u admin:admin  http://localhost:7180/api/v17/cm/service/roleConfigGroups/mgmt-SERVICEMONITOR-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"firehose_heapsize\",\"value\":1073741824}]}" 
curl -X PUT -u admin:admin  http://localhost:7180/api/v17/cm/service/roleConfigGroups/mgmt-HOSTMONITOR-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"firehose_heapsize\",\"value\":1073741824}]}" 

curl -X PUT -u admin:admin  http://localhost:7180/api/v17/cm/service/roleConfigGroups/mgmt-REPORTSMANAGER-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"headlamp_heapsize\",\"value\":268435456}]}" 
curl -X PUT -u admin:admin  http://localhost:7180/api/v17/cm/service/roleConfigGroups/mgmt-EVENTSERVER-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"event_server_heapsize\",\"value\":268435456}]}" 
curl -X PUT -u admin:admin  http://localhost:7180/api/v17/cm/service/roleConfigGroups/mgmt-ALERTPUBLISHER-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"alert_heapsize\",\"value\":268435456}]}" 

## 重启监控服务，使得配置生效
curl -X POST -u admin:admin  http://localhost:7180/api/v17/cm/service/commands/restart
II, 修改cdh服务配置 (调用rest接口)

在这里插入图片描述

## 1，修改zookeeper 客户端最大连接数
curl -X PUT -u admin:admin  http://localhost:7180/api/v17/clusters/cluster_1/services/zookeeper/roleConfigGroups/zookeeper-SERVER-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"maxClientCnxns\",\"value\":300}]}"

## 2，修改yarn NodeManager  堆内存大小
curl -X PUT -u admin:admin  http://localhost:7180/api/v17/clusters/cluster_1/services/yarn/roleConfigGroups/yarn-NODEMANAGER-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"node_manager_java_heapsize\",\"value\":1073741824}]}"


## 3，修改yarn cpu资源 核数总大小
curl -X PUT -u admin:admin  http://localhost:7180/api/v17/clusters/cluster_1/services/yarn/roleConfigGroups/yarn-NODEMANAGER-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"yarn_nodemanager_resource_cpu_vcores\",\"value\":8}]}"

curl -X PUT -u admin:admin  http://localhost:7180/api/v8/clusters/cluster_1/services/yarn/roleConfigGroups/yarn-RESOURCEMANAGER-BASE/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"yarn_scheduler_maximum_allocation_vcores\",\"value\":8}]}"

## 4，修改 hdfs 副本数
curl -X PUT -u admin:admin  http://localhost:7180/api/v17/clusters/cluster_1/services/hdfs/config -H "Content-type: application/json" -d \
"{\"items\":[{\"name\":\"dfs_replication\",\"value\":1}]}"


## 5, 服务自动重启 (hdfs(nn,dn), yarn{nn,rn}, hbase{HM,HR}, solr, hbase-indexer )
curl -X POST -u admin:admin  http://localhost:7180/api/v17/batch -H "Content-type: application/json" -d \
'{
	"items": [{
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/hbase/roleConfigGroups/hbase-HBASETHRIFTSERVER-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		},{
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/hbase/roleConfigGroups/hbase-MASTER-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		}, {
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/hbase/roleConfigGroups/hbase-REGIONSERVER-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		},{
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/yarn/roleConfigGroups/yarn-NODEMANAGER-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		},{
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/yarn/roleConfigGroups/yarn-RESOURCEMANAGER-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		},{
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/ks_indexer/roleConfigGroups/ks_indexer-HBASE_INDEXER-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		},{
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/solr/roleConfigGroups/solr-SOLR_SERVER-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		},{
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/hdfs/roleConfigGroups/hdfs-NAMENODE-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		},{
			"method": "PUT",
			"url": "/api/v8/clusters/cluster/services/hdfs/roleConfigGroups/hdfs-SECONDARYNAMENODE-BASE/config",
			"body": {"items": [{"name": "process_auto_restart","value": "true"}]},
			"contentType": "application/json"
		}
	]
}'
III, 修改cdh服务内存配置 (直接修改mysql数据)
[root@c7-iris2 sh]# cat java_heap.sh
mysql_root_pass=123456
mysql -uroot -p$mysql_root_pass -e 'show databases'

#单位转换
g1_bytes=$((1*1024*1024*1024))
g1_half_bytes=$((g1_bytes/2))
g1_half_half_bytes=$((g1_bytes/2/2))
g2_bytes=$((g1_bytes*2))
g50_mb=$((g1_bytes*50/1024))

#zk
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_half_bytes where attr='zookeeper_server_java_heapsize'"

#hdfs
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_bytes where attr='datanode_java_heapsize'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_bytes where attr='secondary_namenode_java_heapsize'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_bytes where attr='namenode_java_heapsize'"


#hbase
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_bytes where attr='hbase_master_java_heapsize'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_bytes where attr='hbase_regionserver_java_heapsize'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_half_bytes where attr='hbase_thriftserver_java_heapsize'"

#yarn
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_bytes where attr='node_manager_java_heapsize'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_bytes where attr='resource_manager_java_heapsize'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_half_bytes where attr='mr2_jobhistory_java_heapsize'"
#
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g50_mb where attr='yarn_nodemanager_resource_memory_mb'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g50_mb where attr='yarn_scheduler_maximum_allocation_mb'"

#oozie,hive
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_half_half_bytes where attr='oozie_java_heapsize'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_half_bytes where attr='hive_metastore_java_heapsize'"
mysql -uroot -p"$mysql_root_pass" -e "update scm.CONFIGS  set value=$g1_bytes where attr='hiveserver2_java_heapsize'"

## 重启整个集群：使得配置生效
 curl -X POST -u admin:admin  \
 http://localhost:7180/api/v17/clusters/cluster_1/commands/restart \
  -H "Content-type: application/json" \
  -d "{\"restartOnlyStaleServices\":true,\"redeployClientConfiguration\":true}"

在这里插入图片描述

d, 启动/停止服务

1, 整个集群服务

[root@hadoop04 ~]# curl -X POST -u admin:admin http://localhost:7180/api/v17/clusters/Cluster%201/commands/start
{
  "id" : 3096,
  "name" : "Start",
  "startTime" : "2021-02-05T01:39:57.522Z",
  "active" : true,
  "clusterRef" : {
    "clusterName" : "cluster"
  }

2, 集群的单个服务

[root@test-c6 ~]# curl -X POST -u admin:admin \
'http://localhost:7180/api/v1/clusters/Cluster%201/services/hdfs/commands/start'

{
  "id" : 377,
  "name" : "HdfsStartWithFailovers",
  "startTime" : "2020-09-16T11:54:12.652Z",
  "active" : true,
  "serviceRef" : {
    "clusterName" : "cluster",
    "serviceName" : "hdfs"
  }
}

3, cm 监控服务

[root@cdh-c6-master ~]# curl -X POST -u admin:admin \
'http://localhost:7180/api/v17/cm/service/commands/stop' 
{
  "id" : 441,
  "name" : "Stop",
  "startTime" : "2020-10-22T08:30:58.925Z",
  "endTime" : "2020-10-22T08:30:58.925Z",
  "active" : false,
  "success" : false,
  "resultMessage" : "At least one role must be started.",
  "serviceRef" : {
    "serviceName" : "mgmt"
  }
}[root@cdh-c6-master ~]#

3, 查看命令执行结果

############## 查看命令执行结果
[root@test-c6 ~]# curl -u admin:admin 'http://localhost:7180/api/v1/commands/377'
{
  "id" : 377,
  "name" : "HdfsStartWithFailovers",
  "startTime" : "2020-09-16T11:54:12.652Z",
  "endTime" : "2020-09-16T11:54:35.444Z",
  "active" : false,
  "success" : true,
  "resultMessage" : "Successfully started HDFS service",
  "serviceRef" : {
    "clusterName" : "cluster",
    "serviceName" : "hdfs"
  },
  "children" : {
    "items" : [ {
      "id" : 378,
      "name" : "Start",
      "startTime" : "2020-09-16T11:54:12.656Z",
      "endTime" : "2020-09-16T11:54:35.444Z",
      "active" : false,
      "success" : true,
      "resultMessage" : "Successfully started service.",
      "serviceRef" : {
        "clusterName" : "cluster",
        "serviceName" : "hdfs"
      }
    } ]
  }
}

e, 导入集群模板：新建集群

由于模板配置比较复杂，参考ansible-cloudera-manager 安装和配置cdh集群模板的【高级版部分】 https://blog.csdn.net/eyeofeagle/article/details/106121932

1, 集群模板参数

[root@c7-iris2 ~]# cat template.json
{
        "cdhVersion": "5.12.2",
        "cmVersion": "5.12.2",
        "displayName": "cluster_1",
        "products": [
            {
                "product": "CDH",
                "version": "5.12.2-1.cdh5.12.2.p0.4"
            }
        ],

        "services": [
                {
                        "refName": "ZOOKEEPER-1",
                        "roleConfigGroups": [
                            {
                                "base": true,
                                "configs": [],
                                "refName": "ZOOKEEPER-1-SERVER-BASE",
                                "roleType": "SERVER"
                            }
                        ],
                        "serviceConfigs": [
                            {
                                "name": "zookeeper_datadir_autocreate",
                                "value": "true"
                            },
                            {
                                "name": "enableSecurity",
                                "value": "false"
                            }
                        ],
                "serviceType": "ZOOKEEPER"
            }
         ],

        "hostTemplates": [
            {
                "cardinality": 1,
                "refName": "HostTemplate-zk",
                "roleConfigGroupsRefNames": [
                    "ZOOKEEPER-1-SERVER-BASE"
                ]
            }
        ],


        "instantiator": {
            "clusterName": "cluster_1",
            "hosts": [
                {
                    "hostName": "c7-iris2",
                    "hostTemplateRefName": "HostTemplate-zk"
                }
            ],
            "variables": [
            ]
        }
    }
[root@c7-iris2 ~]# curl -X POST  -H "Content-type: application/json" \
 -u admin:admin  http://localhost:7180/api/v17/cm/importClusterTemplate \
 -d @template.json
{
  "id" : 591,
  "name" : "ClusterTemplateImport",
  "startTime" : "2020-10-23T13:52:33.386Z",
  "active" : true,
  "children" : {
    "items" : [ ]
  }
}
[root@c7-iris2 ~]#

2, 命令详情查看

在这里插入图片描述
在这里插入图片描述

2, tsql监控查询

a, 使用指南

http://test-c6:7180/static/help/topics/cm_dg_tsquery.html
在这里插入图片描述

b, 在查询控制台执行sql

http://test-c6:7180/cmf/views/search, 执行sql查询

查询项sql参数
物理内存physical_memory_total, physical_memory_used, physical_memory_memfree, physical_memory_buffers
交换内存swap_total, swap_used, swap_free
负载load_1, load_5, load_15
cpu使用率cpu_percent
网络吞吐速率total_bytes_receive_rate_across_network_interfaces, total_bytes_transmit_rate_across_network_interfaces
磁盘读写速率read_bytes_rate ,write_bytes_rate

在这里插入图片描述

c, 使用api查询sql

[root@test-c6 ~]# curl -u admin:admin  http://localhost:7180/api/v17/timeseries?query=select%20cpu_percent
{
  "items" : [ {
    "timeSeries" : [ {
      "metadata" : {
        "metricName" : "cpu_percent",
        "entityName" : "test-c6",
        "startTime" : "2020-09-16T14:13:45.172Z",
        "endTime" : "2020-09-16T14:18:45.172Z",
        "attributes" : {
          "hostname" : "test-c6",
          "clusterDisplayName" : "Cluster 1",
          "entityName" : "68d905d8-4802-486d-9285-41e70576c05b",
          "clusterName" : "cluster",
          "hostId" : "68d905d8-4802-486d-9285-41e70576c05b",
          "active" : "true",
          "category" : "HOST",
          "version" : "CDH 5.12.0",
          "rackId" : "/default"
        },
        "unitNumerators" : [ "percent" ],
        "unitDenominators" : [ ],
        "expression" : "SELECT cpu_percent WHERE entityName = \"68d905d8-4802-486d-9285-41e70576c05b\" AND category = HOST",
        "metricCollectionFrequencyMs" : 60000,
        "rollupUsed" : "RAW"
      },
      "data" : [ {
        "timestamp" : "2020-09-16T14:14:00.000Z",
        "value" : 5.5,
        "type" : "SAMPLE"
      }, {
        "timestamp" : "2020-09-16T14:15:00.000Z",
        "value" : 3.5,
        "type" : "SAMPLE"
      }, {
        "timestamp" : "2020-09-16T14:16:00.000Z",
        "value" : 3.5,
        "type" : "SAMPLE"
      }, {
        "timestamp" : "2020-09-16T14:17:00.000Z",
        "value" : 3.3,
        "type" : "SAMPLE"
      }, {
        "timestamp" : "2020-09-16T14:18:00.000Z",
        "value" : 3.6,
        "type" : "SAMPLE"
      } ]
    }, {
      "metadata" : {
        "metricName" : "cpu_percent",
        "entityName" : "test-c62",
        "startTime" : "2020-09-16T14:13:45.172Z",
        "endTime" : "2020-09-16T14:18:45.172Z",
        "attributes" : {
          "hostname" : "test-c62",
          "clusterDisplayName" : "Cluster 1",
          "entityName" : "0244fbb4-0d02-40b5-9cac-17d9e388eb97",
          "clusterName" : "cluster",
          "hostId" : "0244fbb4-0d02-40b5-9cac-17d9e388eb97",
          "active" : "true",
          "category" : "HOST",
          "version" : "CDH 5.12.0",
          "rackId" : "/default"
        },
        "unitNumerators" : [ "percent" ],
        "unitDenominators" : [ ],
        "expression" : "SELECT cpu_percent WHERE entityName = \"0244fbb4-0d02-40b5-9cac-17d9e388eb97\" AND category = HOST",
        "metricCollectionFrequencyMs" : 60000,
        "rollupUsed" : "RAW"
      },
      "data" : [ ]
    } ],
    "warnings" : [ ],
    "timeSeriesQuery" : "select cpu_percent"
  } ]
}

3, 使用java调用cm api

发送get/post请求: springboot+RestTemplate

1, pom.xml配置

<dependency>
     <groupId>com.alibaba</groupId>
     <artifactId>fastjson</artifactId>
     <version>1.2.74</version>
 </dependency>
 <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-web</artifactId>
 </dependency>
 <dependency>
     <groupId>org.springframework.boot</groupId>
     <artifactId>spring-boot-starter-tomcat</artifactId>
     <scope>provided</scope> 
 </dependency>

2, 具体代码

在这里插入图片描述

  • http请求工具类
import com.alibaba.fastjson.JSON;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.web.client.RestTemplate;

import javax.xml.bind.DatatypeConverter;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;

public class CmRestUtil {
    private final static Logger LOGGER = LoggerFactory.getLogger(CmRestUtil.class);

    static RestTemplate customRestTemplate;
    static HttpHeaders headers;
    static {
        customRestTemplate = new RestTemplate();
        headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));

        try {
            //username:password--->访问的用户名，密码,并使用base64进行加密，将加密的字节信息转化为string类型
            String format = String.format("%s:%s", "admin", "admin");
            String encoding = DatatypeConverter.printBase64Binary(format.getBytes() );
            headers.set("Authorization", "Basic " + encoding);
        }catch (Exception e){
            throw e;
        }
    }
    
    public static void main(String[] args) throws UnsupportedEncodingException, InterruptedException {
        //1,查看集群信息
        getClusterInfo();
        //2,查看主机信息
        getHostInfo();

        //3,查看服务/角色信息
        getServiceInfo(null);
        //4,启动服务/角色
        startService("hbase");

        //5,tsql监控查询
        timeseriesSqlQuery("select * where roleType=DATANODE");
    }
    public static ResponseEntity<Object> timeseriesSqlQuery(String sql) throws UnsupportedEncodingException {
        Map<String, Object> params = new HashMap<>();
        //String sql="select * where roleType=DATANODE";
        params.put("query",sql);
        org.springframework.http.HttpEntity<Object> requestEntity = new org.springframework.http.HttpEntity<>(params, headers);
        ResponseEntity<Object> response = customRestTemplate.exchange(
                URLDecoder.decode("http://192.168.56.161:7180/api/v17/timeseries", "UTF-8"),
                HttpMethod.POST,
                requestEntity,
                Object.class);

        LOGGER.info("===========code:{},  response {} ", response.getStatusCodeValue(), JSON.toJSONString(response.getBody()));
        return response;
    }
    public static ResponseEntity<Object> startService(String serviceName) throws UnsupportedEncodingException, InterruptedException {
        Map<String, Object> params = new HashMap<>();
        //params.put("query",query);
        org.springframework.http.HttpEntity<Object> requestEntity = new org.springframework.http.HttpEntity<>(params, headers);
        ResponseEntity<Object> response = customRestTemplate.exchange(
                URLDecoder.decode("http://192.168.56.161:7180/api/v1/clusters/Cluster 1/services/"+serviceName+"/commands/start", "UTF-8"),
                HttpMethod.POST,
                requestEntity,
                Object.class);

        LOGGER.info("===========code:{},  response {} ", response.getStatusCodeValue(), JSON.toJSONString(response.getBody()));

        //查看命令执行结果
//        Thread.sleep(2000);
//        response = customRestTemplate.exchange(
//                URLDecoder.decode("http://192.168.56.161:7180/api/v1/commands/593", "UTF-8"),
//                HttpMethod.GET,
//                requestEntity,
//                Object.class);
//        LOGGER.info("command =========== code:{},  response {} ", response.getStatusCodeValue(), JSON.toJSONString(response.getBody()));
        return response;
    }


    public static ResponseEntity<Object> getServiceInfo(String serviceName) throws UnsupportedEncodingException {
        Map<String, Object> params = new HashMap<>();
        //params.put("query",query);
        org.springframework.http.HttpEntity<Object> requestEntity = new org.springframework.http.HttpEntity<>(params, headers);
        /查询所有服务, 查询单个服务
        String url="http://192.168.56.161:7180/api/v17/clusters/Cluster 1/services";
        //         http://192.168.56.161:7180/api/v17/clusters/Cluster 1/services/hbase
        if (serviceName !=null && ! serviceName.equals("") ){
            url+=("/"+serviceName);
        }
        ResponseEntity<Object>  response = customRestTemplate.exchange(
                URLDecoder.decode(url, "UTF-8"),
                HttpMethod.GET,
                requestEntity,
                Object.class);
        LOGGER.info("===========code:{},  response {} ", response.getStatusCodeValue(), JSON.toJSONString(response.getBody()));
        return response;
    }

    public static ResponseEntity<Object> getClusterInfo() throws UnsupportedEncodingException {
        Map<String, Object> params = new HashMap<>();
        //params.put("query",query);
        org.springframework.http.HttpEntity<Object> requestEntity = new org.springframework.http.HttpEntity<>(params, headers);
        ResponseEntity<Object> response = customRestTemplate.exchange(
                URLDecoder.decode("http://192.168.56.161:7180/api/v1/clusters", "UTF-8"),
                HttpMethod.GET,
                requestEntity,
                Object.class);

        LOGGER.info("===========code:{},  response {} ", response.getStatusCodeValue(), JSON.toJSONString(response.getBody()));
        return response;
    }
    public static ResponseEntity<Object> getHostInfo() throws UnsupportedEncodingException {
        Map<String, Object> params = new HashMap<>();
        //params.put("query",query);
        org.springframework.http.HttpEntity<Object> requestEntity = new org.springframework.http.HttpEntity<>(params, headers);
        ResponseEntity<Object> response = customRestTemplate.exchange(
                URLDecoder.decode("http://192.168.56.161:7180/api/v17/hosts", "UTF-8"),
                HttpMethod.GET,
                requestEntity,
                Object.class);
        LOGGER.info("===========code:{},  response {} ", response.getStatusCodeValue(), JSON.toJSONString(response.getBody()));
        return response;
    }
}
  • ApiContorller
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
import web.CmRestUtil;

import java.io.UnsupportedEncodingException;

@Controller
@RequestMapping(value = "/api")
public class CmApiController {
    //1,查看集群信息
    @RequestMapping(value = "/clusterInfo", method = {RequestMethod.GET})
    @ResponseBody
    public Object getClusterInfo() throws UnsupportedEncodingException {
        return CmRestUtil.getClusterInfo();
    }
    //2,查看主机信息
    @RequestMapping(value = "/hostInfo", method = {RequestMethod.GET})
    @ResponseBody
    public Object getHostInfo() throws UnsupportedEncodingException {
        return CmRestUtil.getHostInfo();
    }
    //3,查看服务/角色信息
    @RequestMapping(value = "/serviceInfo", method = {RequestMethod.GET})
    @ResponseBody
    public Object getServiceInfo(@RequestParam(name = "serviceName",required = false) String serviceName) throws UnsupportedEncodingException {
        return CmRestUtil.getServiceInfo(serviceName);
    }
    //4,启动服务/角色
    @RequestMapping(value = "/start", method = {RequestMethod.GET})
    @ResponseBody
    public Object getClusterInfo( @RequestParam(name = "serviceName") String serviceName) throws UnsupportedEncodingException, InterruptedException {
        return CmRestUtil.startService(serviceName);
    }
    //5,tsql监控查询
    @RequestMapping(value = "/tsql", method = {RequestMethod.GET})
    @ResponseBody
    public Object timeseriesSqlQuery( @RequestParam(name = "query") String query) throws UnsupportedEncodingException, InterruptedException {
        return CmRestUtil.timeseriesSqlQuery(query);
    }
}
根哥的博客
关注
  • 1
    点赞
  • 踩
  • 5
    收藏
    觉得还不错? 一键收藏
  • 4
    评论
专栏目录
Cloudera_Hadoop_UserMetrics:查询Cloudera Manager API以生成作业数据以确定用户指标
04-29
Cloudera_Hadoop_UserMetrics 查询Cloudera Manager API以生成作业数据以确定用户指标 查询REST API，如此处所述： : 以写入csv。 这可用于生成自定义指标，以确定集群使用情况。 有关文档，请参见
java调用Cloudera Manager Api实例
03-31
Cloudera Manager API调用实例（JAVA版），已通过测试可用。Cloudera Manager API调用实例（JAVA版），已通过测试可用。
RDMA编程1 建立侦听
热门推荐
winux的专栏
06-28 1万+
RDMA编程 server端开发 Server端如何进行侦听和建立连接
0654-6.2.0-如何通过CM API获取集群事件并入库到MySQL
Hadoop_SC的博客
09-13 340
Fayson的github： https://github.com/fayson/cdhproject 推荐关注微信公众号：“Hadoop实操”，ID：gh_c4c535955d0f 1 文档编写目的 Fayson在本文中介绍如何通过shell 和python 脚本获取CM中重要的告警信息，以便更方便的掌握和分析集群以及集群中节点和服务的健康状况。 测试环境： 1.RedHat7.2 2.CD...
（2）CM.Api项目创建Json包装类与数据库操作接口
Stephanie17395的博客
05-10 428
在 CM.Api 添加文件夹，取名为 Utility，并在 Utility 文件夹下创建类，取名为 ResponseJson(注意大小写) 建立这个类的目的是：该项目前端使用的是 LayUI 的框架，它在进行 AJAX 请求进行数据包装的时候就需要用到 Json 数据，那么ResponseJson 实现的是后端对 Json 数据进行包装转换 ResponseJson 代码： using System; using System.Collections.Generic; using System..
Cloudera Manager API使用(附代码)
weixin_42529806的博客
09-30 4881
Cloudera Manager API使用 前言 1. 需求：如何获取CDH集群的资源使用情况，包括cpu、磁盘、内存 2. 实现：调用CDH提供的Cloudera Manager API包，来实时访问CDH上的各种监控指标数据，如下图 接口介绍 1 获取集群信息 2 获取主机信息 3 获取服务信息 4 获取指标时序数据信息 5 详细信息看：官网文档，使用实例demo：后面 - cloudera官网文档：https://docs.cloudera.com/do...
Cloudera Manager（简称CM）管理大数据平台CDH，CM提供web可视化界面和RESTful API
Fread325的博客
12-27 1265
API列表可以访问： http://localhost:7180/static/apidocs/rest.html https://github.com/cloudera/cm_api 使用方式参考： 1、访问集群概要信息 # curl -u admin:your_password 'http://localhost:7180/api/v19/clusters/' 2、获取集群内服务状态信息 # curl -u admin:your_password 'http://localhost:7
Cloudera Manager API v18
sccdcwc的博客
07-23 2864
介绍 本文档描述了Cloudera Manager REST API。å所有请求和响应都以Javascript对象表示法(JSON)表示。 下面列出的API资源遵循标准的createread-update - delete (CRUD)语义，其中HTTP请求路径定义要执行的实体，而HTTP方法表示要执行的操作类型。 HTTP Method Operation POST Cre...
0629-6.2-如何使用CM API接口获取集群所有节点内存和磁盘使用情况
Hadoop_SC的博客
10-05 1796
Fayson的github： https://github.com/fayson/cdhproject 推荐关注微信公众号：“Hadoop实操”，ID：gh_c4c535955d0f 1 文档编写目的 大家也知道Cloudera Manager提供了丰富的监控指标，部分客户也有自己统一的监控平台，需要将CM上的监控指标集成到统一的监控平台上，因此可以通过提供的API接口获取监控指标，本篇文章F...
通过RESTAPI实现对cdh集群某个进程进行单独的启停
HF0231的博客
08-08 738
通过cm api 单独启动和关闭某台主机的datanode进程。 测试环境 cdh版本 Cloudera Express 5.11.0 需求：停止hadoop03主机上面的datanode进程 具体命令 先获取集群各个主机所对应的hostId 这个可以从数据库里面获取 SELECT name,HOST_IDENTIFIER FROM `scm`.`HOSTS`; 可以看到hadoop0...
cm_api:Cloudera Manager API客户端
05-08
Cloudera Manager RESTful API客户端 是的市场领先的管理平台。 作为业界首个针对Apache Hadoop端到端管理应用程序，Cloudera Manager通过提供对CDH各个部分的细粒度可见性和控制力，为企业部署设定了标准-使运营商...
Cloudera Manager API v32.docx
11-29
Cloudera Manager API v32 Introduced in Cloudera Manager 6.2.0 Cloudera Product Documentation
cloudera manager api
03-28
cloudera manager api
CDH启用kerberos认证
eyeofeagle的博客
01-20 6425
文章目录1，集群架构2, 安装kerberos服务3, CDH集群启用kerberosa, Administrator: securityb, Enable Kerberosc, 确认并勾选d, 填写kdc信息e, 略过CDH管理kerberos, 填写CDH管理员账号4, 使用kerberos认证，调用hive,hbase等服务 1，集群架构 角色 主机ip kerberos软件包 说明...
CDH 安装hue
eyeofeagle的博客
06-06 5277
1, 查看具体报错原因： libmysqlclient_r.so.16: cannot open shared object file: No such file or directory tail -f /opt/cloudera*/logs/cloudera-scm-server/cloudera-scm-server.log ，查看报错信息如下： exec /opt/cloudera/p...
hbase错误排查/数据恢复
eyeofeagle的博客
03-08 5016
1，hbase表：数据结构 2，hbase报错集 由于region发生split, regionserver暂忙，停止了服务，而数据还在往旧的region中插入，导致如下各种错误 ==&amp;amp;amp;amp;amp;amp;amp;amp;amp;gt;解决办法： 第一步： 备份hdfs的表数据（hdfs dfs -mv /hbase/data/default /hbase.back） 第二步： 删除hdfs中的hbase数据目录的全部数据 (hdfs...
CDH 端口未授权访问：hdfs-50070, yarn-8088, jetty漏洞修复
eyeofeagle的博客
10-05 4650
【代码】CDH 端口未授权访问：hdfs-50070, yarn-8088, jetty漏洞修复。
CDH安装: tar包+parcel
eyeofeagle的博客
08-28 3458
修改hdfs超级用户： root cloudera安装hdfs后，默认的配置System User是hdfs, 修改root用户，重启会报错：/var/run/hdfs-sockets权限问题（由于是旧的用户占用了socket,所以删除就行） #java.io.IOException: the path component: '/var/run/hdfs-sockets' is owned ...
CDH集群: 数据恢复/备份
eyeofeagle的博客
01-13 3200
为防止hadoop集群因version等元数据丢失导致启动报错Namenode not format, 而使得所有数据丢失，需要数据业务备份 使用CDH的数据主要为hive, hbase的各表，所以仅需要备份hive,hbase数据（以待hadoop重新格式化后无数据） 测试hdfs重新格式化，数据恢复 说明1： hdfs手动重新格式化后，/目录没有任何数据, 此时需要创建相关的基本目录:（...
cloudera manager安装
最新发布
07-12
Cloudera Manager是一个开源的平台，用于管理和监视Apache Hadoop集群。以下是Cloudera Manager的安装步骤： 1. 安装Java：Cloudera Manager需要Java来运行。确保您的服务器上安装有Java。您可以使用以下命令检查...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 4
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值