Linux下Cassandra集群搭建

1.Cassandra介绍

Cassandra是一套开源分布式NoSQL数据库系统。它最初由Facebook开发，用于储存收件箱等简单格式数据。NoSQl数据库分为4大类，Key-value类型、列存储型、文档类型、图形(Graph)等，而Cassandra为列存储型。
Cassandra的特点：

弹性可扩展性 - Cassandra是高度可扩展的; 它允许添加更多的硬件以适应更多的客户和更多的数据根据要求。
数据存储灵活 - Cassandra适应所有可能的数据格式，包括：结构化，半结构化和非结构化。它可以根据您的需要动态地适应变化的数据结构。
便捷的数据分发 - Cassandra通过在多个数据中心之间复制数据，可以灵活地在需要时分发数据。
快速写入 - Cassandra被设计为在廉价的商品硬件上运行。 它执行快速写入，并可以存储数百TB的数据，而不牺牲读取效率。

2.集群搭建

Cassandra选择了无中心的P2P架构，网络中的所有节点都是对等的，它们构成了一个环，节点之间通过Gossip协议每秒钟和至多三个节点交换一次数据，这样每个节点都拥有其它所有节点的信息，包括位置、状态等。为了保证数据交换的准确性，所有的节点必须使用同一份集群列表，这样的节点又被称作seed节点。
本文将使用3台虚拟机搭建集群：

10.153.54.192
10.153.54.193
10.153.54.194
jdk：jdk1.8.0_161
cassandra：cassandra-3.11

搭建Java运行环境

解压jdk

tar zxvf /usr/local/jdk-8u161-linux-x64.tar.gz -C /usr/java/

配置jdk环境变量文件/etc/profile

[root@indics local]# echo 'export JAVA_HOME=/usr/java/jdk1.8.0_161' >>/etc/profile
[root@indics local]# echo 'export JRE_HOME=${JAVA_HOME}/jre' >>/etc/profile
[root@indics local]# echo 'export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib' >>/etc/profile      
[root@indics local]# echo 'export PATH=${JAVA_HOME}/bin:$PATH' >>/etc/profile
[root@indics local]# source /etc/profile 
[root@indics local]# java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)

安装Cassandra

解压cassandra，后改名为apache-cassandra-prod

[root@indics-api-08 apache-cassandra-prod]# ll
total 544
drwxr-xr-x 2 root root   4096 Jan  7  2014 bin
-rw-r--r-- 1 root root   4832 Dec  9  2020 CASSANDRA-14092.txt
-rw-r--r-- 1 root root 370550 Dec  9  2020 CHANGES.txt
drwxr-xr-x 3 root root   4096 Nov  4 13:29 conf
-rw-r--r-- 1 root root  17986 Feb  3  2021 createks.cql
-rw-r--r-- 1 root root   1834 Feb  3  2021 createks.sh
drwxr-xr-x 5 root root     45 Nov  3 22:00 data
drwxr-xr-x 4 root root     42 May 19 15:41 doc
drwxr-xr-x 2 root root     29 May 19 15:41 interface
drwxr-xr-x 3 root root   4096 May 19 15:41 javadoc
drwxr-xr-x 4 root root   4096 May 19 15:41 lib
-rw-r--r-- 1 root root  11609 Dec  9  2020 LICENSE.txt
drwxr-xr-x 2 root root     62 Nov  3 22:00 logs
-rw-r--r-- 1 root root 113742 Dec  9  2020 NEWS.txt
-rw-r--r-- 1 root root   2811 Dec  9  2020 NOTICE.txt
drwxr-xr-x 3 root root     90 May 19 15:41 pylib
drwxr-xr-x 4 root root   4096 May 19 15:41 tools
[root@indics-api-08 apache-cassandra-prod]#

修改cassandra配置文件 conf/cassandra.yaml

集群名称
修改集群名称:cluster_name: Cassandra-cluster’

#The name of the cluster. This is mainly used to prevent machines in
#one logical cluster from joining another.
cluster_name: 'Cassandra-cluster'

数据存储目录
如果不设置的话，默认在环境变量的目录下；如果设置目录，该目录一定要存在。

#If not set, the default directory is $CASSANDRA_HOME/data/data.
#data_file_directories: 
data_file_directories:
- /data/cassandra/data

提交日志目录

#If not set, the default directory is $CASSANDRA_HOME/data/commitlog.
#commitlog_directory:
#- /data/cassandra/cassandra_commitlog/commitlog
commitlog_directory: 
- /data/cassandra/commitlog

缓存目录

#If not set, the default directory is $CASSANDRA_HOME/data/saved_caches.
#saved_caches_directory: /data/cassandra/saved_caches
saved_caches_directory: /data/cassandra/saved_caches

种子节点
种子节点可配置多节点，中间使用逗号隔开，这里使用 10.153.58.192作为种子节点。
seeds: “10.153.58.192”

seed_provider:
    # Addresses of hosts that are deemed contact points. 
    # Cassandra nodes use this list of hosts to find each other and learn
    # the topology of the ring.  You must change this if you are running
    # multiple nodes!
    - class_name: org.apache.cassandra.locator.SimpleSeedProvider
      parameters:
          # seeds is actually a comma-delimited list of addresses.
          # Ex: "<ip1>,<ip2>,<ip3>"
              - seeds: "10.153.58.192"

监听地址
配置为当前服务器ip，rpc_address 和 listen_address一样填写当前服务器ip

listen_address: 10.153.58.192
broadcast_address: 10.153.58.192
rpc_address: 10.153.58.192
broadcast_rpc_address: 10.153.58.192

修改jvm大小 conf/jvm.options
根据当前虚机内存修改

#It is recommended to set min (-Xms) and max (-Xmx) heap sizes to
#the same value to avoid stop-the-world GC pauses during resize, and
#so that we can lock the heap in memory on startup to prevent any
#of it from being swapped out.
-Xms16G
-Xmx16G

配置cassandra环境变量
vim /etc/profile 添加如下

export CASSANDRA_HOME=/data/cassandra
export PATH=$PATH:$JAVA_HOME/bin:$CASSANDRA_HOME/bin:$SYS_SH/bin
#CQLSH_HOST 本机地址IP
export CQLSH_HOST=10.153.58.192
export CQLSH_PORT=9042

启动集群

#root账户启动命令
./bin/cassaandra -R
#出现如下日志，即启动成功    
18506, 2634169534074999810, 6612719922021534274, -155944934675188714, -2282493852026041530, 6819625517287130096, 3740837535780110746, -7957261670897522845
, 6505751970006179508, -2904834925817926177, 640442872803290314, 515424423439039065, 6018733852448069840, 1990018211401316014, 3186329496090899774, -21652
36144781541727, -6855626646116936353, -5623251552321159850, 2267332447098900413, -7470125253475170004, -4883715623609663199, 8722750050957098252, 41806530
509965111, 7878202023969703432, 2520689853196645755, 8494522861940088094, -9101028288156420668, -1048198256013426364, 2391692976693317011, 837978868054880
9953, 1678335333329287801, 3120837699696315172, 1375302753675295701, 7551493173302415488, -1282663208532909192, -5855948155734809093, 8120945662497963406,
 -2276913016968066643, 2763958832628515411, -8287891182085877261, 1240633823749746019, 4615851547350441986, 3920831583868429363, -1152390796799615052, -44
7582189623056030, -9157698641132676856, 2923533489760496581, -8542723200433815784, 3287763223116597434, -4316416827454696059, 1134866204684812797, 6673325
653943387230, -4032952297351746456, -1015563868884020482, -2558922024917606340, 6738258407875866252, -2452219687274339974, -646258590599762657, 2621500792
343827990, 849352837624378067, -7985833353498175490, 7350025920274563380, 8996914469682605993, 7751010979326527336, 3433409965412539974, 10650013122835065
08, -8477913348162693286, 1844808214862049478, -4763867388504512345, 5076421201406909889, 4181096181277293461, 3692591647422336797, -2659235142433405396, 
5642688194882881166, 5545174181701136922, 5206090484278254058, -2887920926776382631, 2725829207837714672, -8523962104108150722, 4884656965625494082, -9208
293335690690853, -1730939643175634646, -2386460598382340816, 8467463405740643069, 7878554426732827545, -8469755931982633482, 2300473992712988179, -3135572
009036119130, 3020697346159414176, 994836380702900086, -3310819899005358309, 8767314907118743215, -7290495770357446002, -300889276369525275, -190546882504
7896052, -1963347047251061621, -547353654746782933, -2480096603881258471, 7859670493616723409, 434555178269532281, 2492275572859771035, -46743798388799465
20, -7616362614301461788, -7543999509392275954, -2506762804356195264, -6749688044689476512, 5667995576862543844, 4679787675464932702, -3629840514393877711
, 5286737603051878897, -8652984319953046856, 5386525969511421400, -2920322377409788407, 5186325316905785534, 5535516924953793636, 5840851054824218411, 615
7873393114362324, -7003929806696429094, 3721465152441497029, -4555916000028518796, -2110341114204473790, 8925319565271216679, 1600661181621756239, 7552803
832482312550, -3857389259965497295, 3245806889285645243, 1601671475677832408, -4993514195311085670, 6475099582098616485, 4410853712192196017, 443667005253
7343134]
INFO  [main] 2021-11-03 22:01:01,908 StorageService.java:1521 - JOINING: sleeping 30000 ms for pending range setup
INFO  [main] 2021-11-03 22:01:31,908 StorageService.java:1521 - JOINING: Starting to bootstrap...
INFO  [main] 2021-11-03 22:01:32,089 StreamResultFuture.java:90 - [Stream #8c661880-3cae-11ec-ba95-ef68c2800e92] Executing streaming plan for Bootstrap
INFO  [StreamConnectionEstablisher:1] 2021-11-03 22:01:32,092 StreamSession.java:267 - [Stream #8c661880-3cae-11ec-ba95-ef68c2800e92] Starting streaming t
o /10.153.58.193
INFO  [StreamConnectionEstablisher:1] 2021-11-03 22:01:32,098 StreamCoordinator.java:270 - [Stream #8c661880-3cae-11ec-ba95-ef68c2800e92, ID#0] Beginning 
stream session with /10.153.58.193
INFO  [STREAM-IN-/10.153.58.193:7000] 2021-11-03 22:01:32,191 StreamResultFuture.java:187 - [Stream #8c661880-3cae-11ec-ba95-ef68c2800e92] Session with /1
0.153.58.193 is complete
INFO  [StreamConnectionEstablisher:2] 2021-11-03 22:01:32,206 StreamSession.java:267 - [Stream #8c661880-3cae-11ec-ba95-ef68c2800e92] Starting streaming t
o /10.153.58.192 

待所有节点启动完成后, 使用 ./bin/nodetool status 命令查看各个节点状态

[root@indics-api-08 bin]# nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address        Load       Tokens       Owns (effective)  Host ID                               Rack
UN  10.153.58.193  618.82 KiB  256         33.8%             d9de5ed2-3f50-4e6a-a473-d487461d69ba  rack1
UN  10.153.58.192  1023.64 KiB 256         31.1%             e362a33b-6948-40b4-b411-8c46081ce943  rack1
UN  10.153.58.194  1.39 MiB    256         35.1%             c892f966-e7cb-42c6-801b-213233ea16e0  rack1

至此Cassandra集群搭建完毕。

若出现如下问题
nodetool status
nodetool: Failed to connect to ‘127.0.0.1:7199’ - URISyntaxException: ‘Malformed IPv6 address at index 7: rmi://[127.0.0.1]:7199’.

JDK改进的IPv6地址解析存在问题，JDK回退到原来行为，输入以下指令：
nodetool -Dcom.sun.jndi.rmiURLParsing=legacy status