【夜莺监控HBASE记录】

推荐先阅读这篇博客:https://flashcat.cloud/blog/jmx-exporter/

整体流程

1、通过JMX从HBASE拿到prometheus格式的指标数据;
2、配置categraf的prometheus.toml,添加JMX暴露的URL,将数据收集到夜莺指定的时序数据库;
3、然后添加监控大盘和告警规则。

一、配置JMX

1、JMX下载地址
2、修改HBASE启动脚本:hbase-env.sh,在HBASE_MASTER_OPTS和HBASE_REGIONSERVER_OPTS后面添加:-javaagent:jmx的jar的绝对路径=jmx暴露的端口:jmx的config.yaml的绝对路径,例:-javaagent:/opt/jmx/jmx_prometheus_javaagent-0.17.2.jar=27000:/opt/jmx/config.yaml,注意,如果同一台服务器同时存在master和region,注意端口冲突;
3、每个Master、Region Server配置修改后,重启HBASE,验证:访问服务地址,加上JMX暴露端口和/metrics,如果能正确返回数据,则配置成功。

二、categraf配置

1、修改prometheus.toml,添加内容:

[[instances]]
urls = [
   "http://hbase_ip:27000/metrics"
]

url_label_key = "instance"
url_label_value = "{{.Host}}"

2、cd到categraf二进制文件路径,执行./categraf --test --inputs prometheus,检测是否配置成功;

三、夜莺添加监控大盘和告警规则

监控大盘JSON:

{
    "name": "TEST HBase 1.x ",
    "tags": "",
    "ident": "",
    "configs": {
        "version": "3.0.0",
        "links": [],
        "var": [
            {
                "name": "datasource",
                "type": "datasource",
                "definition": "prometheus",
                "defaultValue": 2
            },
            {
                "name": "instance",
                "type": "query",
                "datasource": {
                    "cate": "prometheus",
                    "value": 2
                },
                "definition": "label_values(hadoop_hbase_exceptions, instance)",
                "reg": "",
                "multi": true,
                "allOption": true,
                "allValue": null
            }
        ],
        "panels": [
            {
                "version": "3.0.0",
                "id": "b3f328d2-e4b7-428b-86c0-bf9a46de23da",
                "type": "row",
                "name": "General",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 0,
                    "i": "b3f328d2-e4b7-428b-86c0-bf9a46de23da",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "b201d104-d174-4bd7-bf5d-81c5a2994097",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 1,
                    "i": "b201d104-d174-4bd7-bf5d-81c5a2994097",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": 2,
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_numregionservers{instance=~\"$instance\"}",
                        "legend": "{{pod}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Region Servers",
                "links": [],
                "description": "Number of region servers managed under given namenode instance. Also allows to detect active master rnode change",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "a3c3daed-71d2-4c5f-8fed-b127e7cd9a57",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 1,
                    "i": "a3c3daed-71d2-4c5f-8fed-b127e7cd9a57",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_numdeadregionservers{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Dead Regions",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "7e0751b6-a341-4251-a943-27e864421da3",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 12,
                    "y": 1,
                    "i": "7e0751b6-a341-4251-a943-27e864421da3",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": 2,
                "targets": [
                    {
                        "refId": "B",
                        "expr": "hadoop_hbase_ritcount{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Regions in transition",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "334b0e78-73e8-4ea4-afd7-00fe34e7335d",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 18,
                    "y": 1,
                    "i": "334b0e78-73e8-4ea4-afd7-00fe34e7335d",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_exceptions{instance=~\"$instance\"}[1m])",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Exceptions rate",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "ef8b9efc-a491-430d-812c-236b4d7d584c",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 6,
                    "i": "ef8b9efc-a491-430d-812c-236b4d7d584c",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "B",
                        "expr": "rate(hadoop_hbase_totalrequestcount{instance=~\"$instance\"}[1m])",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Total Requests",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "9ded9d02-dc9a-477f-bae8-3e765895a66e",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 6,
                    "i": "9ded9d02-dc9a-477f-bae8-3e765895a66e",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "B",
                        "expr": "rate(hadoop_hbase_receivedbytes{name=~\".*Master*\",instance=~\"$instance\"}[1m])",
                        "legend": "rx {{ instance }} {{ pod }}"
                    },
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_sentbytes{type=~\".*master.*\",instance=~\"$instance\"}[1m])",
                        "legend": "tx {{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "rx/tx master",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "a8402009-4ab8-45b7-87a5-5a2e89300817",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 12,
                    "y": 6,
                    "i": "a8402009-4ab8-45b7-87a5-5a2e89300817",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "B",
                        "expr": "rate(hadoop_hbase_receivedbytes{name=~\".*RegionServer*\",instance=~\"$instance\"}[1m])",
                        "legend": "rx {{ instance }} {{ pod }}"
                    },
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_sentbytes{type=~\".*region.*\",instance=~\"$instance\"}[1m])",
                        "legend": "tx {{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "rx/tx region",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "f1c52b55-9ba7-4b00-918e-c7ecb7e1bcfe",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 18,
                    "y": 6,
                    "i": "f1c52b55-9ba7-4b00-918e-c7ecb7e1bcfe",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_averageload{instance=~\"$instance\"}",
                        "legend": "{{pod}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Average Load",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "c71fcff5-e0f0-47cf-94bf-c0e317c4c5a7",
                "type": "row",
                "name": "Regions",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 11,
                    "i": "c71fcff5-e0f0-47cf-94bf-c0e317c4c5a7",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "94ee98a0-8747-42d3-95b9-451316ee1c1f",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 12,
                    "i": "94ee98a0-8747-42d3-95b9-451316ee1c1f",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_flushedcellssize{instance=~\"$instance\"}[1m])",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "flushed cells size rate",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "98e4a8d4-5eae-4960-b021-c0b42f3985f6",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 12,
                    "i": "98e4a8d4-5eae-4960-b021-c0b42f3985f6",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "B",
                        "expr": "rate(hadoop_hbase_readrequestcount{instance=~\"$instance\"}[1m])",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "read request rate",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "ad8e5389-c611-4fed-bcab-ef341322baa8",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 12,
                    "y": 12,
                    "i": "ad8e5389-c611-4fed-bcab-ef341322baa8",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "B",
                        "expr": "hadoop_hbase_hlogfilesize{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hlog file size",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "e7b405c7-8635-4d3e-aebc-860c70722689",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 18,
                    "y": 12,
                    "i": "e7b405c7-8635-4d3e-aebc-860c70722689",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "B",
                        "expr": "hadoop_hbase_hlogfilecount{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hlog file count",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "c189a6b1-66f9-42ef-a6be-1fa6bf20af75",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 17,
                    "i": "c189a6b1-66f9-42ef-a6be-1fa6bf20af75",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_flushedcellscount{instance=~\"$instance\"}[1m])",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "flushed cells count rate",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "7afd3ad5-9d24-4c41-9ef9-7378d7216c44",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 17,
                    "i": "7afd3ad5-9d24-4c41-9ef9-7378d7216c44",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_flushedmemstorebytes{instance=~\"$instance\"}[1m])",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "flushed memstore rate",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "fbf88682-1bfe-4787-a16c-e519128055ac",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 12,
                    "y": 17,
                    "i": "fbf88682-1bfe-4787-a16c-e519128055ac",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_storefileindexsize{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "store file index size",
                "links": [],
                "description": "hadoop_hbase_storefileindexsize",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "abf18b4f-64a1-4f29-bec8-a101ba585838",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 18,
                    "y": 17,
                    "i": "abf18b4f-64a1-4f29-bec8-a101ba585838",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_flushedmemstorebytes{instance=~\"$instance\"}[1m])",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "flushed memstore rate",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "2d3ef5f9-ea51-43a7-9b18-75c1e00d907f",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 22,
                    "i": "2d3ef5f9-ea51-43a7-9b18-75c1e00d907f",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_storecount{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "store count",
                "links": [],
                "description": "hadoop_hbase_storecount",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "7cda2090-852d-4a14-b1a7-2b1693b21356",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 22,
                    "i": "7cda2090-852d-4a14-b1a7-2b1693b21356",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_storefilecount{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "store file count",
                "links": [],
                "description": "hadoop_hbase_storefilecount",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "55101a41-b54b-4839-bc25-bf49a6710c41",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 12,
                    "y": 22,
                    "i": "55101a41-b54b-4839-bc25-bf49a6710c41",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_storefilesize{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "store file size",
                "links": [],
                "description": "hadoop_hbase_storefilesize",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "d5ad7e7b-acb8-49f6-8875-735a9380d3b6",
                "type": "row",
                "name": "num_ops",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 27,
                    "i": "d5ad7e7b-acb8-49f6-8875-735a9380d3b6",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "0fa7b25e-3e0c-4f63-a710-ab93ccd10d17",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 0,
                    "y": 28,
                    "i": "0fa7b25e-3e0c-4f63-a710-ab93ccd10d17",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_appendsize_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_appendsize_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": "num_ops",
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "bfc99545-1795-42a8-90a3-990647d13f2d",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 4,
                    "y": 28,
                    "i": "bfc99545-1795-42a8-90a3-990647d13f2d",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_delete_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_delete_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "02d281e4-6476-4a76-b226-b4cfc43d7032",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 8,
                    "y": 28,
                    "i": "02d281e4-6476-4a76-b226-b4cfc43d7032",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_get_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_get_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "5790b117-c7ff-454b-b417-c1ead6d5609b",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 12,
                    "y": 28,
                    "i": "5790b117-c7ff-454b-b417-c1ead6d5609b",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_increment_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_increment_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "cf032a87-1048-498e-860e-6a4dad4fcde6",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 16,
                    "y": 28,
                    "i": "cf032a87-1048-498e-860e-6a4dad4fcde6",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_mutate_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_mutate_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "63279693-f8d6-4991-b5be-4f91d4a9d079",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 20,
                    "y": 28,
                    "i": "63279693-f8d6-4991-b5be-4f91d4a9d079",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_requestsize_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_requestsize_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "78913dd3-de4d-453a-bf60-49f453f779a8",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 0,
                    "y": 33,
                    "i": "78913dd3-de4d-453a-bf60-49f453f779a8",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_responsesize_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_responsesize_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "8ffcaa5d-2280-47f4-bf3b-4f7d4d4311c9",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 4,
                    "y": 33,
                    "i": "8ffcaa5d-2280-47f4-bf3b-4f7d4d4311c9",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_scansize_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_scansize_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "ada66097-5c38-4ec7-b83e-047c71824bdb",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 8,
                    "y": 33,
                    "i": "ada66097-5c38-4ec7-b83e-047c71824bdb",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_splittime_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_splittime_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "bfe05e92-f7ba-4962-ba76-0c3833f0b1da",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 12,
                    "y": 33,
                    "i": "bfe05e92-f7ba-4962-ba76-0c3833f0b1da",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_synctime_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_synctime_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "53f2c7ae-5faf-4e2f-b290-a84243b1dd04",
                "layout": {
                    "h": 5,
                    "w": 4,
                    "x": 16,
                    "y": 33,
                    "i": "53f2c7ae-5faf-4e2f-b290-a84243b1dd04",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_totalcalltime_num_ops{instance=~\"$instance\"}[1m])",
                        "legend": "{{instance}} {{pod}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hadoop_hbase_totalcalltime_num_ops",
                "links": [],
                "description": "Rate of number of operations ",
                "repeat": null,
                "maxPerRow": 6,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "7b5e57b6-3953-4248-a8be-b6237c3af60a",
                "type": "row",
                "name": "HBase Java GC",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 38,
                    "i": "7b5e57b6-3953-4248-a8be-b6237c3af60a",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "19314a32-d4e3-48f3-9717-e2679a987e59",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 39,
                    "i": "19314a32-d4e3-48f3-9717-e2679a987e59",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_gccount{instance=~\"$instance\"}[1m])",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "gc count rate",
                "links": [],
                "description": "hadoop_hbase_gccount rate",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "ba8469d7-6218-4347-999a-1203f96c983c",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 39,
                    "i": "ba8469d7-6218-4347-999a-1203f96c983c",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_gccountparnew{instance=~\"$instance\"}[1m])",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "gc count par new",
                "links": [],
                "description": "hadoop_hbase_gccountparnew",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "1c24f6b3-1fa4-4b72-8c90-c36cc1ac4f2a",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 39,
                    "i": "1c24f6b3-1fa4-4b72-8c90-c36cc1ac4f2a",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_gccountconcurrentmarksweep{instance=~\"$instance\"}[1m])",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "gc countconcurrent mark sweep",
                "links": [],
                "description": "hadoop_hbase_gccountconcurrentmarksweep",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "aa408f9f-05f1-42e6-93c7-72e518d38682",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 45,
                    "i": "aa408f9f-05f1-42e6-93c7-72e518d38682",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_gctimemillis{instance=~\"$instance\"}[1m])",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "gc time",
                "links": [],
                "description": "hadoop_hbase_gctimemillis  rate",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "9204b577-59de-4523-81b5-43070502336e",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 45,
                    "i": "9204b577-59de-4523-81b5-43070502336e",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_gctimemillisparnew{instance=~\"$instance\"}[1m])",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "gc time par new",
                "links": [],
                "description": "hadoop_hbase_gctimemillisparnew  rate",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "e6c9a036-ecdd-472c-b4d3-d56878713cc3",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 45,
                    "i": "e6c9a036-ecdd-472c-b4d3-d56878713cc3",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(hadoop_hbase_gctimemillisconcurrentmarksweep{instance=~\"$instance\"}[1m])",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "gc time conc mark sweep",
                "links": [],
                "description": "hadoop_hbase_gctimemillisconcurrentmarksweep  rate",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "6797d2c4-03b9-40f6-a233-f5db122173e3",
                "type": "row",
                "name": "Heap / non-Heap",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 51,
                    "i": "6797d2c4-03b9-40f6-a233-f5db122173e3",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "7948939c-f07a-4261-aea9-d95b2a37b7a6",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 52,
                    "i": "7948939c-f07a-4261-aea9-d95b2a37b7a6",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "max(hadoop_hbase_memheapusedm{instance=~\"$instance\"})",
                        "legend": "hottest node"
                    },
                    {
                        "refId": "B",
                        "expr": "max(hadoop_hbase_memheapcommittedm{instance=~\"$instance\"})",
                        "legend": "max allowed"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "hottest node - heap used",
                "links": [],
                "description": "green line should be below yellow",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "2c42e7e6-476f-4dae-a062-e160dddb364f",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 52,
                    "i": "2c42e7e6-476f-4dae-a062-e160dddb364f",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_memheapusedm{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ type }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "heap used",
                "links": [],
                "description": "hadoop_hbase_memheapusedm",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "4ff13a94-6ff5-45af-bf77-b4d15ab7f969",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 52,
                    "i": "4ff13a94-6ff5-45af-bf77-b4d15ab7f969",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_memheapcommittedm{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ type }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "heap commited",
                "links": [],
                "description": "hadoop_hbase_memheapcommittedm",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "8de9e795-38c1-42a0-9a20-562098ecdf03",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 18,
                    "y": 52,
                    "i": "8de9e795-38c1-42a0-9a20-562098ecdf03",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_memheapmaxm{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ type }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "heap max",
                "links": [],
                "description": "hadoop_hbase_memheapmaxm",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "60b7ebcf-8009-48cd-bbb0-973e780e042e",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 58,
                    "i": "60b7ebcf-8009-48cd-bbb0-973e780e042e",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_memnonheapusedm{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ type }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "non heap used",
                "links": [],
                "description": "hadoop_hbase_memnonheapusedm",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "fcea6cf4-36a3-43b4-9448-dfa2b52ca99f",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 58,
                    "i": "fcea6cf4-36a3-43b4-9448-dfa2b52ca99f",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_memnonheapcommittedm{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ type }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "non heap commited",
                "links": [],
                "description": "hadoop_hbase_memnonheapcommittedm",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "31741b53-8f98-4978-9f1c-4efada7a66b7",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 58,
                    "i": "31741b53-8f98-4978-9f1c-4efada7a66b7",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_memnonheapcommittedm{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ type }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "non heap max",
                "links": [],
                "description": "hadoop_hbase_memnonheapcommittedm",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "f683497d-57d0-486b-be30-61cb1ec68f1b",
                "type": "row",
                "name": "Java",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 64,
                    "i": "f683497d-57d0-486b-be30-61cb1ec68f1b",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "0776cc8a-d4b4-42cf-9b04-b9df57a8d912",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 65,
                    "i": "0776cc8a-d4b4-42cf-9b04-b9df57a8d912",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_memory_bytes_used{instance=~\"$instance\"}",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_memory_bytes_used",
                "links": [],
                "description": "Used bytes of a given JVM memory area.",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "3f2ee818-bebc-47e7-a5cd-598181313d3e",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 65,
                    "i": "3f2ee818-bebc-47e7-a5cd-598181313d3e",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_memory_bytes_committed{instance=~\"$instance\"}",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_memory_bytes_committed",
                "links": [],
                "description": "Committed (bytes) of a given JVM memory area.",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "76caf56c-a549-49a6-86f1-cc769192fb16",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 65,
                    "i": "76caf56c-a549-49a6-86f1-cc769192fb16",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_memory_bytes_max{instance=~\"$instance\"}",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_memory_bytes_max",
                "links": [],
                "description": "Max (bytes) of a given JVM memory area.",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "d06b55b7-6bbd-4152-ad0c-dfb844a76959",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 18,
                    "y": 65,
                    "i": "d06b55b7-6bbd-4152-ad0c-dfb844a76959",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_memory_bytes_init{instance=~\"$instance\"}",
                        "legend": "{{area}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_memory_bytes_init",
                "links": [],
                "description": "Initial bytes of a given JVM memory area.",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "c370b94f-d3ef-44b9-a299-6541584b0e78",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 71,
                    "i": "c370b94f-d3ef-44b9-a299-6541584b0e78",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_memory_pool_bytes_used{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_memory_pool_bytes_used",
                "links": [],
                "description": "Used bytes of a given JVM memory pool.",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "ff8be2a6-1de5-4922-a1bc-8a1eed84f6fb",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 71,
                    "i": "ff8be2a6-1de5-4922-a1bc-8a1eed84f6fb",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_memory_pool_bytes_committed{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_memory_pool_bytes_committed",
                "links": [],
                "description": "Committed bytes of a given JVM memory pool.",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "0e5ecc7f-5462-45e6-b082-8244486fa07a",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 71,
                    "i": "0e5ecc7f-5462-45e6-b082-8244486fa07a",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_memory_pool_bytes_max{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_memory_pool_bytes_max",
                "links": [],
                "description": "Max bytes of a given JVM memory pool.",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "b76c4ff3-1d5d-4d1e-8f67-354c4e12bcf7",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 18,
                    "y": 71,
                    "i": "b76c4ff3-1d5d-4d1e-8f67-354c4e12bcf7",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_memory_pool_bytes_init{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_memory_pool_bytes_init",
                "links": [],
                "description": "Initial bytes of a given JVM memory pool.",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "45825b04-a703-4c9e-9ec9-e69fe1cf99cb",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 77,
                    "i": "45825b04-a703-4c9e-9ec9-e69fe1cf99cb",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_buffer_pool_capacity_bytes{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_buffer_pool_capacity_bytes",
                "links": [],
                "description": "Bytes capacity of a given JVM buffer pool.\ngauge",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "64baaf5b-af6b-455d-b011-b8cb894d12dc",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 77,
                    "i": "64baaf5b-af6b-455d-b011-b8cb894d12dc",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_buffer_pool_used_buffers{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_buffer_pool_used_buffers",
                "links": [],
                "description": "Used buffers of a given JVM buffer pool.\ngauge",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "d039050c-4002-4682-bf9f-d299ed9f820c",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 77,
                    "i": "d039050c-4002-4682-bf9f-d299ed9f820c",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_threads_started_total{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_threads_started_total",
                "links": [],
                "description": "Started thread count of a JVM\ncounter",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "79b4fdad-7ab4-4fef-9dc5-b91ee7c8ed7e",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 18,
                    "y": 77,
                    "i": "79b4fdad-7ab4-4fef-9dc5-b91ee7c8ed7e",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_threads_daemon{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_threads_daemon",
                "links": [],
                "description": "Daemon thread count of a JVM\ngauge",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "0e549d05-e5a0-4022-bbee-1d444730359d",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 83,
                    "i": "0e549d05-e5a0-4022-bbee-1d444730359d",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_threads_current{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_threads_current",
                "links": [],
                "description": "Current thread count of a JVM\ngauge",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "ad56a95a-2db6-4881-8488-2ab7591e10fa",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 83,
                    "i": "ad56a95a-2db6-4881-8488-2ab7591e10fa",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_threads_peak{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_threads_peak",
                "links": [],
                "description": "Peak thread count of a JVM\ngauge",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "76299b70-7810-4486-a791-52a10a0c7303",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 83,
                    "i": "76299b70-7810-4486-a791-52a10a0c7303",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_threads_deadlocked_monitor{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_threads_deadlocked_monitor",
                "links": [],
                "description": "Cycles of JVM-threads that are in deadlock waiting to acquire object monitors\ngauge",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "732f1a07-1f2d-4228-aa53-45dbe4dee808",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 18,
                    "y": 83,
                    "i": "732f1a07-1f2d-4228-aa53-45dbe4dee808",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "java_lang_memorypool_usage_committed{instance=~\"$instance\"}",
                        "legend": "{{name}} {{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "java_lang_memorypool_usage_committed",
                "links": [],
                "description": "java.lang.management.MemoryUsage (java.lang<type=MemoryPool, name=Code Cache><Usage>committed)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "9c5cf2b6-c350-473d-a589-1fec63a8ae77",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 89,
                    "i": "9c5cf2b6-c350-473d-a589-1fec63a8ae77",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "rate(jvm_threads_started_total{instance=~\"$instance\"}[1m])",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_threads_started_total rate",
                "links": [],
                "description": "Started thread count of a JVM\nrate",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "7d03a5f8-e7a9-4d94-8e35-adb8ec53776c",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 89,
                    "i": "7d03a5f8-e7a9-4d94-8e35-adb8ec53776c",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_threads_deadlocked{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_threads_deadlocked",
                "links": [],
                "description": "Cycles of JVM-threads that are in deadlock waiting to acquire object monitors or ownable synchronizers\ngauge",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "c663a58d-3e6f-477e-93bb-e7ea7b513d7a",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 89,
                    "i": "c663a58d-3e6f-477e-93bb-e7ea7b513d7a",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "java_lang_memorypool_valid{instance=~\"$instance\"}",
                        "legend": "{{name}} {{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "java_lang_memorypool_valid",
                "links": [],
                "description": "Valid (java.lang<type=MemoryPool, name=Code Cache><>Valid)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "4a08c2dc-01f6-4145-838f-89fb42c38056",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 18,
                    "y": 89,
                    "i": "4a08c2dc-01f6-4145-838f-89fb42c38056",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "java_lang_garbagecollector_lastgcinfo_memoryusageaftergc_used{instance=~\"$instance\"}",
                        "legend": "{{name}} {{key}} {{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "java_lang_garbagecollector_lastgcinfo_memoryusageaftergc_used",
                "links": [],
                "description": "java.lang.management.MemoryUsage (java.lang<type=GarbageCollector, name=ParNew, key=Par Survivor Space><LastGcInfo, memoryUsageAfterGc>used)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "04c4925a-a148-4150-8fe1-94d616758e65",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 95,
                    "i": "04c4925a-a148-4150-8fe1-94d616758e65",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "java_lang_memorypool_usagethresholdexceeded{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "java_lang_memorypool_usagethresholdexceeded",
                "links": [],
                "description": "UsageThresholdExceeded (java.lang<type=MemoryPool, name=Code Cache><>UsageThresholdExceeded)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "ebec39d9-114c-4f06-ac88-9a111930f076",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 95,
                    "i": "ebec39d9-114c-4f06-ac88-9a111930f076",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "java_lang_memorypool_collectionusage_used{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "java_lang_memorypool_collectionusage_used",
                "links": [],
                "description": "java.lang.management.MemoryUsage (java.lang<type=MemoryPool, name=Par Eden Space><CollectionUsage>used)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "ffda576f-1980-49ed-a342-2d471b9bbda5",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 12,
                    "y": 95,
                    "i": "ffda576f-1980-49ed-a342-2d471b9bbda5",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "java_lang_memorypool_usage_used{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "java_lang_memorypool_usage_used",
                "links": [],
                "description": "java.lang.management.MemoryUsage (java.lang<type=MemoryPool, name=Code Cache><Usage>used)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "066ac18c-8a5f-46bd-a268-5f5c73fd86f1",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 18,
                    "y": 95,
                    "i": "066ac18c-8a5f-46bd-a268-5f5c73fd86f1",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "java_lang_memorypool_peakusage_committed{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "java_lang_memorypool_peakusage_committed",
                "links": [],
                "description": "java.lang.management.MemoryUsage (java.lang<type=MemoryPool, name=Code Cache><PeakUsage>committed)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "45b59bd3-00f1-407d-a181-bf929c07482b",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 0,
                    "y": 101,
                    "i": "45b59bd3-00f1-407d-a181-bf929c07482b",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "java_lang_memorypool_collectionusagethresholdexceeded{instance=~\"$instance\"}",
                        "legend": "{{pool}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "java_lang_memorypool_collectionusagethresholdexceeded",
                "links": [],
                "description": "CollectionUsageThresholdExceeded (java.lang<type=MemoryPool, name=Par Eden Space><>CollectionUsageThresholdExceeded)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "2365e0ea-b1c6-4b2b-a074-ad30f47dc249",
                "layout": {
                    "h": 6,
                    "w": 6,
                    "x": 6,
                    "y": 101,
                    "i": "2365e0ea-b1c6-4b2b-a074-ad30f47dc249",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "jvm_threads_state{instance=~\"$instance\"}",
                        "legend": "{{state}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "jvm_threads_state",
                "links": [],
                "description": "CollectionUsageThresholdExceeded (java.lang<type=MemoryPool, name=Par Eden Space><>CollectionUsageThresholdExceeded)",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "4f530bbd-8622-40c4-b65f-f04c040415fd",
                "type": "row",
                "name": "Memstore",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 107,
                    "i": "4f530bbd-8622-40c4-b65f-f04c040415fd",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "129af5c3-7eda-4516-a5c8-e7d8f9344876",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 108,
                    "i": "129af5c3-7eda-4516-a5c8-e7d8f9344876",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(hbase_metric_memstoresize{instance=~\"$instance\"})",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "memstore size total",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "8913f073-5e66-475d-ac6c-312f936b9ff2",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 108,
                    "i": "8913f073-5e66-475d-ac6c-312f936b9ff2",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hadoop_hbase_memstoresize{instance=~\"$instance\"}",
                        "legend": "{{ instance }} {{ pod }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "memstore size",
                "links": [],
                "description": "hadoop_hbase_memstoresize",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "8100cbf9-ec79-48ed-8d61-7924979a0865",
                "type": "row",
                "name": "Compactions",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 113,
                    "i": "8100cbf9-ec79-48ed-8d61-7924979a0865",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "10521540-e37b-480f-88c2-69c0c75708a5",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 114,
                    "i": "10521540-e37b-480f-88c2-69c0c75708a5",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_compactionscompletedcount{instance=~\"$instance\"}[1m]))",
                        "legend": ""
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Compactions completed rate",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "87f7639e-c101-4aca-9615-1d1ab261f746",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 114,
                    "i": "87f7639e-c101-4aca-9615-1d1ab261f746",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_compactionscompletedcount{instance=~\"$instance\"}[1m])) by (instance)",
                        "legend": "{{pod}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Compactions completed by instance",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "cd6ae39f-068d-493b-8d30-ec704318ef5a",
                "type": "timeseries",
                "name": "Compactions completed by table",
                "description": "",
                "links": [],
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 12,
                    "y": 114,
                    "i": "cd6ae39f-068d-493b-8d30-ec704318ef5a",
                    "isResizable": true
                },
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_compactionscompletedcount{instance=~\"$instance\",table=~\"$table\",region=~\"$region\"}[1m])) by (instance)",
                        "legend": "{{pod}} {{instance}}"
                    }
                ],
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "version": "3.0.0",
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "fillOpacity": 0.5,
                    "stack": "off"
                },
                "maxPerRow": 4
            },
            {
                "version": "3.0.0",
                "id": "c2e1a08a-c6d1-4693-89cf-68f264bba685",
                "type": "timeseries",
                "name": "Bytes compacted",
                "description": "",
                "links": [],
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 119,
                    "i": "c2e1a08a-c6d1-4693-89cf-68f264bba685",
                    "isResizable": true
                },
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_numbytescompactedcount{instance=~\"$instance\",table=~\"$table\",region=~\"$region\"}[1m]))",
                        "legend": ""
                    }
                ],
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "version": "3.0.0",
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "fillOpacity": 0.5,
                    "stack": "off"
                },
                "maxPerRow": 4
            },
            {
                "version": "3.0.0",
                "id": "8b2043be-5d05-4d94-84df-111938a48859",
                "type": "timeseries",
                "name": "Bytes compacted by instance",
                "description": "",
                "links": [],
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 119,
                    "i": "8b2043be-5d05-4d94-84df-111938a48859",
                    "isResizable": true
                },
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_numbytescompactedcount{instance=~\"$instance\",table=~\"$table\",region=~\"$region\"}[1m])) by (instance)",
                        "legend": "{{table}} {{instance}}"
                    }
                ],
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "version": "3.0.0",
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "fillOpacity": 0.5,
                    "stack": "off"
                },
                "maxPerRow": 4
            },
            {
                "version": "3.0.0",
                "id": "fa9ac65b-444f-4e47-9050-c1f6630c31fd",
                "type": "timeseries",
                "name": "Bytes comacted by table",
                "description": "",
                "links": [],
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 12,
                    "y": 119,
                    "i": "fa9ac65b-444f-4e47-9050-c1f6630c31fd",
                    "isResizable": true
                },
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_numbytescompactedcount{instance=~\"$instance\",table=~\"$table\",region=~\"$region\"}[1m])) by (table)",
                        "legend": "{{table}} {{instance}}"
                    }
                ],
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "version": "3.0.0",
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "fillOpacity": 0.5,
                    "stack": "off"
                },
                "maxPerRow": 4
            },
            {
                "version": "3.0.0",
                "id": "9fe14d8e-d3aa-44c3-9b53-e7e5f8d66af8",
                "type": "timeseries",
                "name": "Files compacted rate",
                "description": "",
                "links": [],
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 124,
                    "i": "9fe14d8e-d3aa-44c3-9b53-e7e5f8d66af8",
                    "isResizable": true
                },
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_numfilescompactedcount{instance=~\"$instance\",table=~\"$table\",region=~\"$region\"}[1m]))",
                        "legend": ""
                    }
                ],
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "version": "3.0.0",
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "fillOpacity": 0.5,
                    "stack": "off"
                },
                "maxPerRow": 4
            },
            {
                "type": "timeseries",
                "id": "6877c5a4-3f1a-4165-a345-39a7fd5f4ef9",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 124,
                    "i": "6877c5a4-3f1a-4165-a345-39a7fd5f4ef9",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_numfilescompactedcount{instance=~\"$instance\",table=~\"$table\",region=~\"$region\"}[1m])) by (instance)",
                        "legend": "{{pod}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Files compacted by instance",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "47070f52-cb60-4299-bff8-4229791ab694",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 12,
                    "y": 124,
                    "i": "47070f52-cb60-4299-bff8-4229791ab694",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hbase_metric_numfilescompactedcount{instance=~\"$instance\"}[1m])) by (instance)",
                        "legend": "{{pod}} {{instance}}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Files compacted by instance",
                "links": [],
                "description": "",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "938e88c6-2eb4-4718-9b4a-b23dd67ddc28",
                "type": "row",
                "name": "Major Compactions",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 129,
                    "i": "938e88c6-2eb4-4718-9b4a-b23dd67ddc28",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "4880f88f-bbf9-4b93-b8b1-0fc80916f882",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 130,
                    "i": "4880f88f-bbf9-4b93-b8b1-0fc80916f882",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hadoop_hbase_majorcompactedcellscount{instance=~\"$instance\"}[1m]))",
                        "legend": ""
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Major compacted cells count",
                "links": [],
                "description": "rate",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "828f2fa5-4b50-4aad-b545-96c8f833b8ba",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 130,
                    "i": "828f2fa5-4b50-4aad-b545-96c8f833b8ba",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hadoop_hbase_majorcompactedcellscount{instance=~\"$instance\"}[1m])) by (instance)",
                        "legend": "{{ instance }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Major compacted cells count /  instance",
                "links": [],
                "description": "rate ",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "00484cb9-cc35-498a-bf0b-80aa3e477bce",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 135,
                    "i": "00484cb9-cc35-498a-bf0b-80aa3e477bce",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hadoop_hbase_majorcompactedcellssize{instance=~\"$instance\"}[1m]))",
                        "legend": ""
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Major compacted cells size",
                "links": [],
                "description": "rate",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "type": "timeseries",
                "id": "e7b5b580-db59-4cde-a5cf-596bb8dd9856",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 6,
                    "y": 135,
                    "i": "e7b5b580-db59-4cde-a5cf-596bb8dd9856",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "sum(rate(hadoop_hbase_majorcompactedcellssize{instance=~\"$instance\"}[1m])) by (instance)",
                        "legend": "{{ instance }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "Major compacted cells size / instance",
                "links": [],
                "description": "rate",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            },
            {
                "version": "3.0.0",
                "id": "c7a98810-ae04-45d4-a1f7-847220a220c4",
                "type": "row",
                "name": "Memstore per Region (SUPER SLOW)",
                "collapsed": true,
                "layout": {
                    "h": 1,
                    "w": 24,
                    "x": 0,
                    "y": 140,
                    "i": "c7a98810-ae04-45d4-a1f7-847220a220c4",
                    "isResizable": false
                },
                "panels": []
            },
            {
                "type": "timeseries",
                "id": "b903d462-17ca-44dd-bfc6-39976fb272c5",
                "layout": {
                    "h": 5,
                    "w": 6,
                    "x": 0,
                    "y": 141,
                    "i": "b903d462-17ca-44dd-bfc6-39976fb272c5",
                    "isResizable": true
                },
                "version": "3.0.0",
                "datasourceCate": "prometheus",
                "datasourceValue": "${datasource}",
                "targets": [
                    {
                        "refId": "A",
                        "expr": "hbase_metric_memstoresize{instance=~\"$instance\",table=~\"$table\",region=~\"$region\"}",
                        "legend": "{{ instance }} {{ region }}"
                    }
                ],
                "transformations": [
                    {
                        "id": "organize",
                        "options": {}
                    }
                ],
                "name": "memstore size per region",
                "links": [],
                "description": "hadoop_hbase_memstoresize",
                "maxPerRow": 4,
                "options": {
                    "tooltip": {
                        "mode": "all",
                        "sort": "none"
                    },
                    "legend": {
                        "displayMode": "hidden"
                    },
                    "standardOptions": {},
                    "thresholds": {
                        "steps": [
                            {
                                "color": "#634CD9",
                                "value": null,
                                "type": "base"
                            }
                        ]
                    }
                },
                "custom": {
                    "drawStyle": "lines",
                    "lineInterpolation": "linear",
                    "spanNulls": false,
                    "lineWidth": 1,
                    "fillOpacity": 0.5,
                    "gradientMode": "none",
                    "stack": "off",
                    "scaleDistribution": {
                        "type": "linear"
                    }
                }
            }
        ]
    }
}

告警规则(根据自己服务配置)

100 * (avg_over_time(jvm_memory_bytes_used{area="heap",instance="ip:27000"}[1m])/  max_over_time(jvm_memory_bytes_max{area="heap",instance="ip:27000"}[1m]))  > 80,内存使用率大于80%;
rate(hadoop_hbase_gccount[1m])  > 0.25,每分钟GC次数超过15次
hadoop_hbase_storefilesize/1024/1024/1024  > 500,存储大小超过500G
hadoop_hbase_numdeadregionservers  > 0,存在dead region
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值