Monitor ClickHouse with Prometheus & Grafana

1. Monitor ClickHouse with Prometheus & Grafana

Prometheus is a 9-year-old, 100K-line GoLang project designed for systems monitoring. There have been almost 600 contributors to the project with Fabian Reinartz of Google and Julius Volz, formerly of Google and now the Founder of PromLabs, leading in commit counts.

Grafana is an interactive visualization web app. The underlying codebase is made up of ~500K lines of GoLang and Typescript. 1.4K developers have contributed to the project in its 8-year history. The project’s roots can be found in Stockholm with Torkel Ödegaard having contributed the largest number of commits to the project. In the past few years, he’s commercialised support with Grafana Labs, a firm based in New York City with almost 400 members of staff as of this writing.

1.1. Installing ClickHouse

The following commands were run on a fresh install of Ubuntu 20.04 LTS on a system with 16 GB of RAM.

$ sudo apt-key adv \
    --keyserver hkp://keyserver.ubuntu.com:80 \
    --recv E0C56BD4
$ echo "deb http://repo.yandex.ru/clickhouse/deb/stable/ main/" | \
    sudo tee /etc/apt/sources.list.d/clickhouse.list
$ sudo apt update

I’ll install ClickHouse.

$ sudo apt install \
    clickhouse-client \
    clickhouse-server-common

The following will enable Prometheus metrics exposure in ClickHouse’s config.

$ sudo vi /etc/clickhouse-server/config.xml

Remove the XML comments around the tags. It should look like the following:

<prometheus>
    <