ClickHouse v22.8.5.29-lts 安装笔记

欢迎关注【云原生生态圈】订阅号

部署日期参与人员部署目标版本
Mon 19 Sep 2022@devops1. 部署一套本地化的分布式持久存储的Clickhouse集群v0.1

集群信息

集群节点信息

主机名IP地址Clickhouse节点信息
bigdata110.0.10.195clickhouse-node1
bigdata210.0.10.194clickhouse-node2
Bigdata310.0.10.196clickhouse-node3

节点基础信息

操作系统内核版本Docker版本Clickhouse版本安装方式
Ubuntu 20.04.3 LTS5.4.0-91-generic20.10.12v22.8.5.29-ltsApt包管理器

集群端口预留信息

端口说明备注
8123http端口
9000tcp端口

集群配置信息

配置项配置信息
服务端口- 8123: http端口
- 9000: tcp端口
数据目录/data/clickhouse_bigdata
集群名称bigdata_3shards_1replicas
配置文件路径集群配置:
- 默认配置文件:/etc/clickhouse-server/config.xml
- zookeeper配置文件: /etc/clickhouse-server/config.d/zookeeper.xml
- 宏配置文件: /etc/clickhouse-server/config.d/macros.xml
- 集群配置切片副本配置文件: /etc/clickhouse-server/config.d/remote-servers.xml
用户授权认证配置:
- 管理员clickhouse_manager/etc/clickhouse-server/users.d/clickhouse_manager.xml
- 普通用户clickhouse_operator: /etc/clickhouse-server/users.d/clickhouse_operator.xml
日志目录/var/log/clickhouse-server/
分布式集群模式3shards_1replicas
服务管理sudo clickhouse [start/stop]
负载均衡地址- HTTP: clickhouse.devopsman.cn
- TCP: clickhouse-tcp.devopsman.cn

安装Clickhouse服务

sudo apt-get update
sudo apt-get install -y apt-transport-https ca-certificates dirmngr
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 8919F6BD2B48D754

echo "deb https://packages.clickhouse.com/deb stable main" | sudo tee \
    /etc/apt/sources.list.d/clickhouse.list
sudo apt-get update

apt-cache madison clickhouse-server | head -n5

sudo apt-get install -y clickhouse-server=22.8.5.29 clickhouse-client=22.8.5.29

# 修改clickhouse的家目录以及相关属主组
mkdir /home/clickhouse

chown -R clickhouse.clickhouse /home/clickhouse

usermod -d /home/clickhouse clickhouse

# 准备clickhouse的数据存储目录
mkdir /data/clickhouse_bigdata

chown -R clickhouse.clickhouse /data/clickhouse_bigdata

# 在clickhouse的默认配置文件中修改clickhouse的数据目录
sed -e 's#/var/lib/clickhouse#/data/clickhouse_bigdata#g' /etc/clickhouse-server/config.xml |grep -i '/data/clickhouse_bigdata'

sed -i 's#/var/lib/clickhouse#/data/clickhouse_bigdata#g' /etc/clickhouse-server/config.xml |grep -i '/data/clickhouse_bigdata'

# sudo service clickhouse-server start
# clickhouse-client # or "clickhouse-client --password" if you've set up a password.

配置节点域名解析

⚠️ 每个节点都需要配置

  • 本地host配置
cat >> /etc/hosts << EOF
10.0.10.195 cknode1.devopsman.cn bigdata1 
10.0.10.194 cknode2.devopsman.cn bigdata2
10.0.10.196 cknode3.devopsman.cn bigdata3
EOF
  • DNS解析平台上进行解析

ClickHouse 的集群配置

配置切片和副本remote-servers.xml

<clickhouse>
    <default_on_cluster_name>bigdata</default_on_cluster_name>
    <default_replica_path>/clickhouse/tables/{cluster}/{shard}/{database}/{table}</default_replica_path>
    <default_replica_name>{replica}</default_replica_name>
    <remote_servers>
        <!-- User-specified clusters -->
        <bigdata_3shards_1replicas>
            <shard>
                <replica>
                    <host>cknode1.devopsman.cn</host>
                    <port>9000</port>
                </replica>
            </shard>
            <shard>
                <replica>
                    <host>cknode2.devopsman.cn</host>
                    <port>9000</port>
                </replica>
            </shard>
            <shard>
                <replica>
                    <host>cknode3.devopsman.cn</host>
                    <port>9000</port>
                </replica>
            </shard>
        </bigdata_3shards_1replicas>
    </remote_servers>
</clickhouse>

配置集群需要的Zookeeper信息zookeeper.xml

<clickhouse>
    <zookeeper>
        <node>
            <host>zookeeper-1.devopsman.cn</host>
            <port>2181</port>
        </node>
        <node>
            <host>zookeeper-2.devopsman.cn</host>
            <port>2181</port>
        </node>
        <node>
            <host>zookeeper-3.devopsman.cn</host>
            <port>2181</port>
        </node>
        <session_timeout_ms>30000</session_timeout_ms>
        <operation_timeout_ms>10000</operation_timeout_ms>
    </zookeeper>
    <distributed_ddl>
        <path>/clickhouse/bigdata/task_queue/ddl</path>
    </distributed_ddl>
</clickhouse>

宏macros.xml

<clickhouse>
    <macros>
        <installation>bigdata</installation>
        <cluster>bigdata</cluster>
        <shard>2</shard>
	<replica>1</replica>
    </macros>
</clickhouse>

监听配置信息listen.xml

<clickhouse>
    <listen_try>1</listen_try>
    <listen_host>0.0.0.0</listen_host>
    <listen_host>127.0.0.1</listen_host>
</clickhouse>

ClickHouse 账号认证授权

管理员账号:clickhouse_manager.xml

<clickhouse>
    <users>
        <clickhouse_manager>
            <access_management>1</access_management>
            <networks>
                <ip>10.0.10.0/24</ip>
                <ip>127.0.0.1</ip>
                <ip>::/0</ip>
            </networks>
            <password_sha256_hex>ff3ebe0b384ba70a48ef866d0e4e8bfcc3104709f3858d95e51111111110e2e683cbb3da4</password_sha256_hex>
            <profile>clickhouse_manager</profile>
            <quota>default</quota>
        </clickhouse_manager>
    </users>
    <profiles>
        <clickhouse_manager>
            <log_queries>1</log_queries>
            <skip_unavailable_shards>1</skip_unavailable_shards>
            <http_connection_timeout>10</http_connection_timeout>
            <parallel_view_processing>1</parallel_view_processing>
        </clickhouse_manager>
    </profiles>
</clickhouse>

维护账号: clickhouse_operator.xml

<clickhouse>
    <users>
        <clickhouse_operator>
            <networks>
                <ip>10.0.10.0/24</ip>
                <ip>127.0.0.1</ip>
                <ip>::/0</ip>
            </networks>
            <password_sha256_hex>716b36073a90c6fe1d445ac1af85f4777c5b7a155cea35911111111111111</password_sha256_hex>
            <profile>clickhouse_operator</profile>
            <quota>default</quota>
        </clickhouse_operator>

    </users>
    <profiles>
        <clickhouse_operator>
            <log_queries>1</log_queries>
            <skip_unavailable_shards>1</skip_unavailable_shards>
            <http_connection_timeout>10</http_connection_timeout>
        </clickhouse_operator>
    </profiles>
</clickhouse>

在所有的配置文件配置完成后,切记要修改xml文件的所属属性

chown clickhouse.clickhouse /etc/clickhouse-server/config.d/*.xml /etc/clickhouse-server/users.d/*.xml

FQA

  • 安装日志
Creating clickhouse group if it does not exist.
groupadd -r clickhouse
Creating clickhouse user if it does not exist.
useradd -r --shell /bin/false --home-dir /nonexistent -g clickhouse clickhouse
Will set ulimits for clickhouse user in /etc/security/limits.d/clickhouse.conf.
Creating config directory /etc/clickhouse-server/config.d that is used for tweaks of main server configuration.
Creating config directory /etc/clickhouse-server/users.d that is used for tweaks of users configuration.
Config file /etc/clickhouse-server/config.xml already exists, will keep it and extract path info from it.
/etc/clickhouse-server/config.xml has /var/lib/clickhouse/ as data path.
/etc/clickhouse-server/config.xml has /var/log/clickhouse-server/ as log path.
Users config file /etc/clickhouse-server/users.xml already exists, will keep it and extract users info from it.
Creating log directory /var/log/clickhouse-server/.
Creating data directory /var/lib/clickhouse/.
Creating pid directory /var/run/clickhouse-server.
chown -R clickhouse:clickhouse '/var/log/clickhouse-server/'
chown -R clickhouse:clickhouse '/var/run/clickhouse-server'
chown  clickhouse:clickhouse '/var/lib/clickhouse/'
groupadd -r clickhouse-bridge
useradd -r --shell /bin/false --home-dir /nonexistent -g clickhouse-bridge clickhouse-bridge
chown -R clickhouse-bridge:clickhouse-bridge '/usr/bin/clickhouse-odbc-bridge'
chown -R clickhouse-bridge:clickhouse-bridge '/usr/bin/clickhouse-library-bridge'
Enter password for default user: # 》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》》此处是配置default的密码
Password for default user is saved in file /etc/clickhouse-server/users.d/default-password.xml.
Setting capabilities for clickhouse binary. This is optional.
chown -R clickhouse:clickhouse '/etc/clickhouse-server'

ClickHouse has been successfully installed.

Start clickhouse-server with:
sudo clickhouse start

Start clickhouse-client with:
clickhouse-client --password

Synchronizing state of clickhouse-server.service with SysV service script with /lib/systemd/systemd-sysv-install.
Executing: /lib/systemd/systemd-sysv-install enable clickhouse-server
Created symlink /etc/systemd/system/multi-user.target.wants/clickhouse-server.service → /lib/systemd/system/clickhouse-server.service.
Setting up clickhouse-client (22.8.5.29) ...
Processing triggers for systemd (245.4-4ubuntu3.13) ...
  • 服务无法启动,通过debug方式启动clickhouse服务查看debug日志
sudo -u clickhouse clickhouse-server --config-file=/etc/clickhouse-server/config.xml

参考信息

  • 0
    点赞
  • 1
    收藏
  • 打赏
    打赏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
©️2022 CSDN 皮肤主题:数字20 设计师:CSDN官方博客 返回首页
评论

打赏作者

云原生生态圈

你的鼓励是我创作的动力!

¥2 ¥4 ¥6 ¥10 ¥20
输入1-500的整数
余额支付 (余额:-- )
扫码支付
扫码支付:¥2
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、C币套餐、付费专栏及课程。

余额充值