Scylla3.0.4在CentOS7.4上的安装

介绍

ScyllaDB是用C++重写的Cassandra，其官网宣称其每节点每秒可处理100万TPS。ScyllaDB完全兼容Apache Cassandra，拥有比Cassandra多10X倍的吞吐量，并降低了延迟。ScyllaDB是性能优异的NoSQL列存储数据库。

 

安装准备

• Scylla支持CentOS 7.3或以上64位x86_64的版本
• 由于ABRT会与Scylla的coredump配置冲突，安装前需要删除ABRT

sudo yum remove -y abrt

• 检查一下内核，建议升级内核至3.15以上

Linux kernels before 3.15 did not have good asynchronous append support, which is required by Scylla

•  配置EPEL源 https://fedoraproject.org/wiki/EPEL

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

•  在Scylla官网注册以便获得下载链接

sudo curl -o /etc/yum.repos.d/scylla.repo -L http://repositories.scylladb.com/scylla/repo/93495f33-6092-45c3-a5cb-718bd237e641/centos/scylladb-3.0.repo

安装Scylla

yum install scylla

如有Liblz4.so.1()(64bit) 依赖包报错，可手工在https://altlinux.pkgs.org/sisyphus/classic-x86_64/liblz4-1.8.3-alt2.x86_64.rpm.html下载安装

安装Scylla所需的包列表

-rw-r--r--  1 root root  1074616 3月  19 14:09 cryptopp-5.6.2-10.el7.x86_64.rpm
-rw-r--r--  1 root root    15080 3月  19 10:20 epel-release-latest-7.noarch.rpm
-rw-r--r--  1 root root   693128 3月  19 14:09 gnutls-3.3.26-9.el7.x86_64.rpm
-rw-r--r--  1 root root    34016 3月  19 14:09 gnutls-c++-3.3.26-9.el7.x86_64.rpm
-rw-r--r--  1 root root   149060 3月  19 14:09 hwloc-1.11.2-2.el7.x86_64.rpm
-rw-r--r--  1 root root  1565752 3月  19 14:09 hwloc-libs-1.11.2-2.el7.x86_64.rpm
-rw-r--r--  1 root root    83972 3月  19 14:09 jsoncpp-0.10.5-2.el7.x86_64.rpm
-rw-r--r--  1 root root    39672 3月  19 14:09 libreport-filesystem-2.1.11-38.el7.x86_64.rpm
-rw-r--r--  1 root root    50128 3月  19 14:09 libtool-ltdl-2.4.2-22.el7_3.x86_64.rpm
-rw-r--r--  1 root root    56120 3月  19 14:09 libyaml-0.1.4-11.el7_0.x86_64.rpm
-rw-r--r--  1 root root   100668 3月  19 14:04 lz4-1.7.5-2.el7.x86_64.rpm
-rw-r--r--  1 root root   438740 3月  19 14:09 mdadm-4.0-5.el7.x86_64.rpm
-rw-r--r--  1 root root   335092 3月  19 14:09 nettle-2.7.1-8.el7.x86_64.rpm
-rw-r--r--  1 root root    95532 3月  19 14:09 pciutils-3.5.1-2.el7.x86_64.rpm
-rw-r--r--  1 root root   346608 3月  19 14:09 protobuf-2.5.0-8.el7.x86_64.rpm
-rw-r--r--  1 root root    96200 3月  19 14:09 pyparsing-1.5.6-9.el7.noarch.rpm
-rw-r--r--  1 root root    52336 3月  19 14:09 python34-3.4.9-2.el7.x86_64.rpm
-rw-r--r--  1 root root  8663672 3月  19 14:09 python34-libs-3.4.9-2.el7.x86_64.rpm
-rw-r--r--  1 root root   153374 3月  19 14:09 python34-PyYAML-3.11-3.el7.x86_64.rpm
-rw-r--r--  1 root root    33307 3月  19 14:09 python34-six-1.11.0-1.el7.noarch.rpm
-rw-r--r--  1 root root    79168 3月  19 14:09 python3-pyudev-0.21.0-1.el7.noarch.rpm
-rw-r--r--  1 root root     5980 3月  19 14:09 python-backports-1.0-8.el7.x86_64.rpm
-rw-r--r--  1 root root    12204 3月  19 14:09 python-backports-ssl_match_hostname-3.4.0.2-4.el7.noarch.rpm
-rw-r--r--  1 root root   232336 3月  19 14:09 python-chardet-2.2.1-1.el7_1.noarch.rpm
-rw-r--r--  1 root root    95740 3月  19 14:09 python-requests-2.6.0-1.el7_1.noarch.rpm
-rw-r--r--  1 root root   406464 3月  19 14:09 python-setuptools-0.9.8-7.el7.noarch.rpm
-rw-r--r--  1 root root    29516 3月  19 14:09 python-six-1.9.0-2.el7.noarch.rpm
-rw-r--r--  1 root root   103764 3月  19 14:09 python-urllib3-1.10.2-3.el7.noarch.rpm
-rw-r--r--  1 root root   669372 3月  19 14:09 python-urwid-1.1.1-3.el7.x86_64.rpm
-rw-r--r--  1 root root   157008 3月  19 14:09 PyYAML-3.10-11.el7.x86_64.rpm
-rw-r--r--  1 root root     2280 3月  19 14:09 scylla-3.0.4-0.20190313.5e3a52024.el7.x86_64.rpm
-rw-r--r--  1 root root    17080 3月  19 14:09 scylla-conf-3.0.4-0.20190313.5e3a52024.el7.x86_64.rpm
-rw-r--r--  1 root root     3156 3月  19 14:09 scylla-env-1.1-1.el7.noarch.rpm
-rw-r--r--  1 root root  5918116 3月  19 14:09 scylla-jmx-3.0.4-20190313.1191757.el7.noarch.rpm
-rw-r--r--  1 root root     3316 3月  19 14:09 scylla-kernel-conf-3.0.4-0.20190313.5e3a52024.el7.x86_64.rpm
-rw-r--r--  1 root root    38240 3月  19 14:09 scylla-libatomic73-7.3.1-1.2.el7.centos.x86_64.rpm
-rw-r--r--  1 root root    93292 3月  19 14:09 scylla-libgcc73-7.3.1-1.2.el7.centos.x86_64.rpm
-rw-r--r--  1 root root   469824 3月  19 14:09 scylla-libstdc++73-7.3.1-1.2.el7.centos.x86_64.rpm
-rw-r--r--  1 root root 23139484 3月  19 14:09 scylla-server-3.0.4-0.20190313.5e3a52024.el7.x86_64.rpm
-rw-r--r--  1 root root   157072 3月  19 14:09 scylla-tools-3.0.4-20190313.254550e5e6.el7.noarch.rpm
-rw-r--r--  1 root root 28700876 3月  19 14:09 scylla-tools-core-3.0.4-20190313.254550e5e6.el7.noarch.rpm
-rw-r--r--  1 root root  1056052 3月  19 14:09 thrift-0.9.1-15.el7.x86_64.rpm
-rw-r--r--  1 root root   296360 3月  19 14:09 trousers-0.3.14-2.el7.x86_64.rpm

配置Scylla

配置/etc/scylla/scylla.yaml文件

此文件类似cassandra的cassandra.yaml
cluster_name：集群名称，一个友好的名称，比如命名“Scylla Cluster”；
seeds：种子节点，为了使集群中的各节点在启动时能发现其他节点，需要指定种子节点（seeds），各节点都先和种子节点通信，通过种子节点获取其他节点列表，然后和其他节点通信。种子节点可以指定多个，用逗号分割开。
storage_port：可以使用默认的配置，7000，这个端口用于接收命令和数据。
listen_address：这个地址用于节点之间的通信，也可以配置listen_interface，指定使用哪个网卡接口。两者选一配置即可，不要同时配置。
native_transport_port：这个端口用于Client到Cassandra服务的通信。注意配置防火墙。
start_rpc: true
rpc_address: 改成本节点地址

配置Scylla-server

# scylla_setup

[root@localhost scylla]# scylla_setup
Skip any of the following steps by answering 'no'
Do you want to run check your kernel version?
Yes - runs a  script to verify that the kernel for this instance qualifies to run Scylla. No - skips the kernel check.
[YES/no]yes
INFO  2019-03-19 14:49:30,208 [shard 0] iotune - /var/tmp/mnt passed sanity checks
This is a supported kernel version.
Do you want to verify the ScyllaDB packages are installed?
Yes - runs a script to confirm that ScyllaDB is installed. No - skips the installation check.
[YES/no]no
Do you want the Scylla server service to automatically start when the Scylla node boots?
Yes - Scylla server service automatically starts on Scylla node boot. No - skips this step. Note you will have to start the Scylla Server service manually.
[YES/no]yes
Created symlink from /etc/systemd/system/multi-user.target.wants/scylla-server.service to /usr/lib/systemd/system/scylla-server.service.
Unable to retrieve version information
Do you want to disable SELinux?
Yes - disables SELinux. Choosing Yes greatly improves performance. No - keeps SELinux activated.
[YES/no]yes
Do you want to setup Network Time Protocol(NTP) to auto-synchronize the current time on the node?
Yes - enables time-synchronization. This keeps the correct time on the node. No - skips this step.
[YES/no]no       
Do you want to setup RAID0 and XFS?
It is recommended to use RAID0 and XFS for Scylla data. If you select yes, you will be prompted to choose the unmounted disks to use for Scylla data. Selected disks are formatted as part of the process.
Yes - choose a disk/disks to format and setup for RAID0 and XFS. No - skip this step.
[YES/no]yes
Are you sure you want to setup RAID0 and XFS?
If you choose Yes, the selected drive will be reformated, erasing all existing data in the process.
[YES/no]no
Do you want to enable coredumps?
Yes - sets up coredump to allow a post-mortem analysis of the Scylla state just prior to a crash. No - skips this step.
[YES/no]yes
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e"
Do you want to setup a system-wide customized configuration for Scylla?
Yes - setup the sysconfig file. No - skips this step.
[YES/no]yes
Do you want to enable Network Interface Card (NIC) and disk(s) optimization?
Yes - optimize the NIC queue and disks settings. Selecting Yes greatly improves performance. No - skip this step.
[YES/no]yes
ERROR: timed out. Your system can't be tuned until the issue is fixed.
Traceback (most recent call last):
  File "/usr/lib/scylla/scylla_sysconfig_setup", line 75, in <module>
    rps_cpus = out('{} --tune net --nic {} --get-cpu-mask'.format(perftune_base_command(), ifname))
  File "/usr/lib/scylla/scylla_util.py", line 280, in out
    return subprocess.check_output(cmd, shell=shell).strip().decode('utf-8')
  File "/usr/lib64/python3.4/subprocess.py", line 617, in check_output
    raise CalledProcessError(retcode, process.args, output=output)
subprocess.CalledProcessError: Command '['/usr/lib/scylla/perftune.py', '--tune', 'disks', '--dir', '/var/lib/scylla/data', '--dir', '/var/lib/scylla/commitlog', '--tune', 'net', '--nic', 'eth0', '--get-cpu-mask']' returned non-zero exit status 1
NIC queue setup failed. Press any key to continue...

Do you want iotune to study your disks IO profile and adapt Scylla to it?
Yes - let iotune study my disk(s). Note that this action will take a few minutes. No - skip this step.
[YES/no]yes
tuning /sys/devices/virtual/block/dm-0
tuning: /sys/devices/virtual/block/dm-0/queue/nomerges 2
tuning /sys/devices/pci0000:00/0000:00:04.0/virtio1/host2/target2:0:0/2:0:0:0/block/sda/sda2
tuning /sys/devices/pci0000:00/0000:00:04.0/virtio1/host2/target2:0:0/2:0:0:0/block/sda
tuning: /sys/devices/pci0000:00/0000:00:04.0/virtio1/host2/target2:0:0/2:0:0:0/block/sda/queue/nomerges 2
INFO  2019-03-19 14:51:22,239 [shard 0] iotune - /var/lib/scylla/data passed sanity checks
WARN  2019-03-19 14:51:22,240 [shard 0] iotune - Scheduler for /sys/devices/pci0000:00/0000:00:04.0/virtio1/host2/target2:0:0/2:0:0:0/block/sda/queue/scheduler set to cfq. It is recommend to set it to noop before evaluation so as not to skew the results.
Starting Evaluation. This may take a while...
Measuring sequential write bandwidth: 109 MB/s
Measuring sequential read bandwidth: 336 MB/s
Measuring random write IOPS: 4538 IOPS
Measuring random read IOPS: 285 IOPS
Recommended --num-io-queues: 1
Writing result to /etc/scylla.d/io_properties.yaml
Writing result to /etc/scylla.d/io.conf
Do you want to install node exporter to export Prometheus data from the node? Note that the Scylla monitoring stack uses this data
Yes - install node exporter. No - skip this  step.
[YES/no]yes

node_exporter already installed
node exporter setup failed. Press any key to continue...

Do you want to set the CPU scaling governor to Performance level on boot?
Yes - sets the CPU scaling governor to performance level. No - skip this step.
[YES/no]yes
This computer doesn't supported CPU scaling configuration.
Do you want to enable fstrim service?
Yes - runs fstrim on your SSD. No - skip this step.
[YES/no]yes
ScyllaDB setup finished.

启动Scylla

systemctl start scylla-server

查看集群状态

nodetool status

连接集群

cqlsh 10.10.10.101

注意：如果配置了rpc地址为实际地址，则默认连接使用的127.0.0.1不会启动服务监听，必须写上集群服务地址

模拟压测

cassandra-stress 用于压力测试 模拟写入和读取  

cassandra-stress write -mode cql3 native -node 10.10.10.101

cassandra-stress write n=10000 插入一万条数据

cassandra-stress read n=10000 读取一万条数据  

cassandra-stress read duration=3m 持续三分钟一直读取

创建keyspace

cqlsh> create keyspace domestic with replication = {'class':'SimpleStrategy','replication_factor':2};

cqlsh> desc domestic;

CREATE KEYSPACE domestic WITH replication = {'class': 'SimpleStrategy', 'replication_factor': '2'}  AND durable_writes = true;

cqlsh> use domestic;
cqlsh:domestic> desc keyspaces;

system_schema  domestic  keyspace1           system_traces
system_auth    system    system_distributed

创建table

cqlsh> use domestic;
cqlsh:domestic> create table student (id int primary key, name text,age int);

cqlsh:domestic> desc tables;

student

cqlsh:domestic> insert into student (id,name,age) values (1,'aa',20);
cqlsh:domestic> insert into student (id,name,age) values (2,'bb',19);
cqlsh:domestic> select * from student;

 id | age | name
----+-----+------
  1 |  20 |   aa
  2 |  19 |   bb

(2 rows)

cqlsh:domestic>  quit