【Clickhouse_docker-copose】安装部署

【Clickhouse docker-compose 安装部署】

一、基础资源

硬件资源:3台centos7.8虚拟机

软件资源:docker-24.0.1,docker-compose-2.18.1,zookeeper3.8,clickhouse-23

配置信息:自行配置/etc/hosts 的ip与别名映射

二、分布式集群，docker-compose.yaml编写

1.vi docker-compose.yaml

version: '2'
services:
  clickhouse:
    image: docker.io/bitnami/clickhouse:23
    environment:
      - ALLOW_EMPTY_PASSWORD=yes
    network_mode: "host"
    volumes:
    #配置文件的目录
      - ./etc:/opt/bitnami/clickhouse/etc/ 
      - ./clickhouse_data:/bitnami/clickhouse
    depends_on:
      - zookeeper
  zookeeper:
    image: docker.io/bitnami/zookeeper:3.8
    restart: always
    #开启host网络模式保证zk的网络畅通
    network_mode: "host"
    volumes:
    #创建相对路径
      - "./zookeeper/zookeeper_data:/bitnami"
    environment: 
      - ALLOW_ANONYMOUS_LOGIN=yes
      #zoo_id,每个节点需要更换
      - ZOO_SERVER_ID=1
      - ZOO_LISTEN_ALLIPS_ENABLED=yes
      - ZOO_SERVERS=zk1:2888:3888,zk2:2888:3888,zk3:2888:3888

volumes:
  clickhouse_data:
    driver: local

注意:分布式下需要修改ZOO_SERVER_ID的id号,起始id必须为1,且有顺序。

2. 运行docker-compose

#运行,拉取镜像，并自动创建zookeeper、clickhouse_data、etc等目录
$ docker-compose up

3.赋权限

#需要给目录赋权限，否则容器中无法会权限报错
$ chown -R 1001:1001 ./zookeeper
$ chown -R 1001:1001 ./clickhouse_data
$ chown -R 1001:1001 ./etc

三、配置文件

1.从运行中的Clickhouse容器中获取config.xml、users.xml配置文件

#root进入容器,查看配置文件在哪
$ docker exec -ti -u root clickhouse-容器id bash
#记录配置文件位置，退出容器然后将文件拷贝出来到宿主机./etc目录下
$ docker cp fa20edc65b74(容器id):/etc/clickhouse-server/config.xml ./etc
$ docker cp fa20edc65b74():/etc/clickhouse-server/users.xml ./etc

2.修改config.xml文件

配置文件中配置集Clickhouse、zookeeper群信息及分片、副本信息，在文本中间增加如下配置

<zookeeper>
    <node index="1">
        <host>zk1</host>
        <port>2181</port>
    </node>
    <node index="2">
        <host>zk2</host>
        <port>2181</port>
    </node>
    <node index="3">
        <host>zk3</host>
        <port>2181</port>
    </node>
</zookeeper>
#3分片，2副本
<remote_servers>
    <ck_cluster>
        <shard>
            <weight>1</weight>
            <internal_replication>true</internal_replication>
            <replica>
                <host>ck1</host>
                <port>9000</port>
            </replica>
            <replica>
                <host>ck2</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <weight>1</weight>
            <internal_replication>true</internal_replication>
            <replica>
                <host>ck2</host>
                <port>9000</port>
            </replica>
            <replica>
                <host>ck3</host>
                <port>9000</port>
            </replica>
        </shard>
        <shard>
            <weight>1</weight>
            <internal_replication>true</internal_replication>
            <replica>
                <host>ck3</host>
                <port>9000</port>
            </replica>
            <replica>
                <host>ck1</host>
                <port>9000</port>
            </replica>
        </shard>
    </ck_cluster>
</remote_servers>

<macros>
#分片id与上对应
    <shard>01</shard>    
#下列配置副本名称每个节点都不一样
    <replica>rep_1_2</replica>
</macros>

3.重新运行docker-compose up

4.集群验证

#使用连接工具连接clickhouse任意节点，运行以下sql,查看集群概况
$ select * from system.clusters;

四、注意事项

需要开启防火墙对应的端口

#开启8123,2181,2888,3888等端口,
$ firewall-cmd --add-port=8123/tcp --permanent
$ firewall-cmd --add-port=2181/tcp --permanent
$ firewall-cmd --add-port=2888/tcp --permanent
$ firewall-cmd --add-port=3888/tcp --permanent
$ firewall-cmd --reload

参考:clickhouse分区操作实践_clickhouse多级分区_逃跑的沙丁鱼的博客-CSDN博客