clickhouse集群部署说明

clickhouse集群部署说明（5分片，分片内1副本）

集群架构：

分片1：机器hiido-clickhouse-01的9000端口

分片2：机器hiido-clickhouse-02的9000端口

分片3：机器hiido-clickhouse-03的9000端口

分片4：机器hiido-clickhouse-04的9000端口

分片5：机器hiido-clickhouse-05的9000端口

 

1.部署机器列表，使用了5台物理机器

分别给每台机器配置hostname，并为每台服务器添加域名

root@ubuntu:~# hostnamectl set-hostname hiido-clickhouse-01

10.12.17.12 hiido-clickhouse-01.hiido.host.yydevops.com
10.12.14.88 hiido-clickhouse-02.hiido.host.yydevops.com
10.12.14.89 hiido-clickhouse-03.hiido.host.yydevops.com
10.12.14.155 hiido-clickhouse-04.hiido.host.yydevops.com
10.12.14.156 hiido-clickhouse-05.hiido.host.yydevops.com

 

2.部署zookeeper集群

在zookeeper官网下载安装包apache-zookeeper-3.6.2-bin.tar.gz

https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/zookeeper-3.6.2/apache-zookeeper-3.6.2-bin.tar.gz

上传压缩包到目录/root，解压到目录/usr/local/apache-zookeeper-3.6.2-bin，并建立软链接/usr/local/zookeeper

root@hiido-clickhouse-01:~# tar -zxf apache-zookeeper-3.6.2-bin.tar.gz

root@hiido-clickhouse-01:~# mv apache-zookeeper-3.6.2-bin /usr/local/

root@hiido-clickhouse-01:~# ln -s /usr/local/apache-zookeeper-3.6.2-bin /usr/local/zookeeper

root@hiido-clickhouse-01:~# cp /usr/local/zookeeper/conf/zoo_sample.cfg /usr/local/zookeeper/conf/zoo.cfg

root@hiido-clickhouse-01:~# vim /usr/local/zookeeper/conf/zoo.cfg

/usr/local/zookeeper/conf/zoo.cfg

... dataDir=/data/zookeeper dataLogDir=/data/logs/zookeeper ... autopurge.purgeInterval=0 globalOutstandingLimit=200 server.1=hiido-clickhouse-01.hiido.host.yydevops.com:2888:3888 server.2=hiido-clickhouse-02.hiido.host.yydevops.com:2888:3888 server.3=hiido-clickhouse-03.hiido.host.yydevops.com:2888:3888

设置myid

hiidoagent@hiido-clickhouse01:/data# vim /data/zookeeper/myid  #hiido-clickhouse-01为1，hiido-clickhouse-02为2，hiido-clickhouse-03为3

启动zookeeper服务，并查看zookeeper服务状态

hiidoagent@hiido-clickhouse-01:/usr/local/zookeeper$ bin/zkServer.sh start

hiidoagent@hiido-clickhouse-01:/usr/local/zookeeper$ bin/zkServer.sh status

使用客户端连接测试

hiidoagent@hiido-clickhouse-01:/usr/local/zookeeper# /usr/local/zookeeper/bin/zkCli.sh -timeout 5000 -server hiido-clickhouse-01.hiido.host.yydevops.com:2181

说明：这里使用了zookeeper的默认端口:2181，后面配clickhouse集群的时候需要使用到该端口

 

3.到clickhouse官网下载clickhouse相应版本的安装包

https://repo.yandex.ru/clickhouse/deb/stable/main/

clickhouse-common-static_20.9.3.45_amd64.deb

clickhouse-server_20.9.3.45_all.deb

clickhouse-client_20.9.3.45_all.deb

 

4.安装clickhouse

root@hiido-clickhouse-01:~# dpkg -i clickhouse-common-static_20.9.3.45_amd64.deb

root@hiido-clickhouse-01:~# dpkg -i clickhouse-server_20.9.3.45_all.deb

root@hiido-clickhouse-01:~# dpkg -i clickhouse-client_20.9.3.45_all.deb

说明：安装clickhouse-server过程中设置了默认用户的密码是clickhouse，后面客户端连接需要使用到该密码，配置集群互访也需要使用到

 

--------------------------------------------------------------------------------------------------

查看已安装的deb软件包

root@hiido-clickhouse-01:~# dpkg -l

ii clickhouse-client 20.9.3.45 all Client binary for ClickHouse
ii clickhouse-common-static 20.9.3.45 amd64 Common files for ClickHouse
ii clickhouse-server 20.9.3.45 all Server binary for ClickHouse

--------------------------------------------------------------------------------------------------

如果要卸载clickhouse，可以执行删除deb软件包，连同配置文件一起删除

root@hiido-clickhouse-01:~# dpkg -r clickhouse-client

root@hiido-clickhouse-01:~# dpkg --purge clickhouse-client

root@hiido-clickhouse-01:~# dpkg -r clickhouse-server

root@hiido-clickhouse-01:~# dpkg --purge clickhouse-server

root@hiido-clickhouse-01:~# dpkg -r clickhouse-common-static

root@hiido-clickhouse-01:~# dpkg --purge clickhouse-common-static

root@hiido-clickhouse-01:~# rm -rf /etc/clickhouse-client

root@hiido-clickhouse-01:~# rm -rf /etc/clickhouse-server

 

5.修改clickhouse配置文件config.xml的日志文件路径、数据存储路径

root@hiido-clickhouse-01:/data1# mkdir -p clickhouse/log/clickhouse-server

root@hiido-clickhouse-01:/data1# chown clickhouse:clickhouse -R /data1/clickhouse/log

root@hiido-clickhouse-01:/data1# vim /etc/clickhouse-server/config.xml

/etc/clickhouse-server/config.xml

...         <log>/data1/clickhouse/log/clickhouse-server/clickhouse-server.log</log>         <errorlog>/data1/clickhouse/log/clickhouse-server/clickhouse-server.err.log</errorlog> ...     <!-- Path to data directory, with trailing slash. -->     <path>/data1/clickhouse/</path>     <!-- Path to temporary data for processing hard queries. -->     <tmp_path>/data1/clickhouse/tmp/</tmp_path>     <!-- Directory with user provided files that are accessible by 'file' table function. -->     <user_files_path>/data1/clickhouse/user_files/</user_files_path>     <!-- Directory in <clickhouse-path> containing schema files for various input formats.          The directory will be created if it doesn't exist.       -->     <format_schema_path>/data1/clickhouse/format_schemas/</format_schema_path>

说明：因为磁盘/data1空间最大，有19T

 

6.配置clickhouse集群

root@hiido-clickhouse-01:/data1# vim /etc/metrika.xml

/etc/metrika.xml

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73

<yandex>     <clickhouse_remote_servers>         <perftest_5shards_1replicas><!-- 自定义集群名称 -->             <shard>                 <internal_replication>true</internal_replication>                 <replica>                     <host>hiido-clickhouse-01.hiido.host.yydevops.com</host>                     <port>9000</port>                 </replica>             </shard>             <shard>                 <internal_replication>true</internal_replication>                 <replica>                     <host>hiido-clickhouse-02.hiido.host.yydevops.com</host>                     <port>9000</port>