搭建zookeeper+clickhouse

weixin_38627000

已于 2022-09-07 17:54:10 修改

阅读量988

点赞数 1

分类专栏： centos7 文章标签： java-zookeeper zookeeper linux

于 2022-08-31 16:39:03 首次发布

本文链接：https://blog.csdn.net/weixin_38627000/article/details/126626709

版权

centos7 专栏收录该内容

30 篇文章 2 订阅

订阅专栏

三台环境

10.99.33.251 clickhouse01
10.99.33.252 clickhouse02
10.99.33.253 clickhouse03

以下步骤在三台机器上都需要操作

vim /etc/hosts
10.99.33.251 clickhouse01.hmtech.com
10.99.33.252 clickhouse02.hmtech.com
10.99.33.253 clickhouse03.hmtech.com

下载&解压&转移zookeeper包

wget https://dlcdn.apache.org/zookeeper/zookeeper-3.7.1/apache-zookeeper-3.7.1-bin.tar.gz
tar xf apache-zookeeper-3.7.1-bin.tar.gz
mkdir /apps
mv apache-zookeeper-3.7.1-bin /apps/
cd cd /apps/apache-zookeeper-3.7.1-bin/conf/
cp zoo_sample.cfg zoo.cfg

编辑zoo.cfg文件

vim /apps/apache-zookeeper-3.7.1-bin/conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/apps/apache-zookeeper-3.7.1-bin/data/zookeeper
dataLogDir=/apps/apache-zookeeper-3.7.1-bin/log/zookeeper
clientPort=2182
autopurge.snapRetainCount=0

server.251=10.99.33.251:2888:3888
server.252=10.99.33.252:2888:3888
server.253=10.99.33.253:2888:3888

创建数据目录

mkdir {data,log}
mkdir data/zookeeper

在每台服务器上分配创建myid文件

[root@clickhouse01 ~]# cat /apps/apache-zookeeper-3.7.1-bin/data/zookeeper/myid
251

[root@clickhouse02 ~]# cat /apps/apache-zookeeper-3.7.1-bin/data/zookeeper/myid
252

[root@clickhouse03 ~]# cat /apps/apache-zookeeper-3.7.1-bin/data/zookeeper/myid
253

先删除本机现有的Java&安装java

yum -y remove java && yum install fontconfig java-11-openjdk

配置环境变量

cd /usr/lib/jvm/java-11-openjdk-11.0.16.0.8-1.el7_9.x86_64/

vim /etc/profile
export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-11.0.16.0.8-1.el7_9.x86_64
export CLASSPATH=.: $JAVA_HOME/lib/dt.jar:$ JAVA_HOME/lib/tools.jar
export PATH= $P A T H :$ JAVA_HOME/bin:$PATH

source /etc/profile

启动zookeeper

./bin/zkServer.sh start

配置zookeeper服务自启

[root@clickhouse01 ~]# cat /etc/rc.d/init.d/zookeeper
#!/bin/bash
#chkconfig:2345 20 90
#description:zookeeper
#processname:zookeeper
case $1 in
start)
sh /apps/apache-zookeeper-3.7.1-bin/bin/zkServer.sh start
;;
stop)
sh /apps/apache-zookeeper-3.7.1-bin/bin/zkServer.sh stop
;;
status)
sh /apps/apache-zookeeper-3.7.1-bin/bin/zkServer.sh status
;;
restart)
sh /apps/apache-zookeeper-3.7.1-bin/bin/zkServer.sh restart
;;
*)
echo “Usage:$0(start|stop|status|restart)”
;;
esac
exit
!
在这里插入图片描述

添加权限

chmod +x zookeeper

service zookerper start && service zookerper status

加入启动项

chkconfig --add zookeeper && chkconfig --list

转载请附上本文链接或出处

搭建clickhouse

grep -q sse4_2 /proc/cpuinfo && echo “SSE 4.2 supported” || echo “SSE 4.2 not supported”
sudo yum install -y yum-utils
sudo yum-config-manager --add-repo https://packages.clickhouse.com/rpm/clickhouse.repo
sudo yum install -y clickhouse-server clickhouse-client
yum -y install libicu libicu-devel
rpm -qa | grep clickhouse
cp /etc/clickhouse-server/config.xml /etc/clickhouse-server/config.xml.bak

systemctl start clickhouse-server
systemctl status clickhouse-server

单机已搭建完成

构建集群
再每台机器上新建/etc/metrika.xml

<yandex>
  <clickhouse_remote_servers>
    <!--自定义集群名称-->
    <perftest_3shards_1replicas>
      <!--定义集群的分片数量，3个shard标签说明有3个节点-->
      <shard>
         <internal_replication>true</internal_replication>
         <!--定义分片的副本数量，这里副本有3个-->
         <replica>
            <host>10.99.33.251</host>
            <port>9000</port>
         </replica>
      </shard>
      <shard>
         <internal_replication>true</internal_replication>
         <replica>
            <host>10.99.33.252</host>
            <port>9000</port>
         </replica>
      </shard>
      <shard>
         <internal_replication>true</internal_replication>
         <replica>
            <host>10.99.33.253</host>
            <port>9000</port>
         </replica>
      </shard>
    </perftest_3shards_1replicas>
  </clickhouse_remote_servers>

  <!--zookeeper相关配置-->
  <zookeeper-servers>
    <node index="251">
       <host>10.99.33.251</host>
       <port>2182</port>
    </node>
    <node index="252">
       <host>10.99.33.252</host>
       <port>2182</port>
    </node>
    <node index="253">
       <host>10.99.33.253</host>
       <port>2182</port>
     </node>
  </zookeeper-servers>

  <macros>
      <replica>10.99.33.251</replica>
  </macros>

  <networks>
      <ip>::/0</ip>
  </networks>

  <clickhouse_compression>
      <case>
         <min_part_size>10000000000</min_part_size>
         <min_part_size_ratio>0.01</min_part_size_ratio>
         <method>lz4</method>
      </case>
  </clickhouse_compression>

</yandex>

注意：为什么直接vim /etc/metrika.xml一个新文件，这里很难去理解，有点莫名其妙，其实如果仔细看过clickhouse的配置文件/etc/clickhouse-server/config.xml就能明白，有这么一段被注释的配置说明：
<!- - If element has ‘incl’ attribute, then for it’s value will be used corresponding substitution from another file.
By default, path to file with substitutions is /etc/metrika.xml. It could be changed in config in ‘include_from’ element.
Values for substitutions are specified in /yandex/name_of_substitution elements in that file.
–>

配置一个节点完成之后，将/etc/metrika.xml scp到其他节点。
并各自修改

节点IP地址/域名

重启clickhouse-server
[root@clickhouse01 ~]# systemctl restart clickhouse-server
[root@clickhouse01 ~]# systemctl status clickhouse-server