CentOS7.4安装部署crateDB集群

虚拟机环境

主机名	IP
manager.learn.bigdata	192.168.183.101	节点1
master.learn.bigdata	192.168.183.102	节点2
worker.learn.bigdata	192.168.183.103	节点3

安装包

下载地址：https://cdn.crate.io/downloads/releases/
我的下载版本：https://cdn.crate.io/downloads/releases/crate-4.1.2.tar.gz
下载后将安装包上传到所有节点/opt目录（自定义）下，解压。

[root@manager opt]# ll
total 71780
-rw-r--r-- 1 root root 56412465 Jul 13 19:39 crate-4.1.2.tar.gz
[root@manager opt]# tar -zxvf crate-4.1.2.tar.gz
[root@manager opt]# ll
total 71780
drwxr-xr-x 7 root root      135 Feb 14 22:56 crate-4.1.2
-rw-r--r-- 1 root root 56412465 Jul 13 19:39 crate-4.1.2.tar.gz

crateDB4.1.2版本的目录结构如下

[root@manager crate-4.1.2]# ll
total 32
drwxr-xr-x 2 root root    95 Jul 13 19:40 bin
-rw-r--r-- 1 root root  1124 Feb 14 22:54 CHANGES.txt
drwxr-xr-x 2 root root    48 Jul 13 19:40 config
drwxr-xr-x 3 root root  4096 Jul 13 19:40 lib
-rw-r--r-- 1 root root 14982 Feb 14 22:54 LICENSE
drwxr-xr-x 2 root root     6 Feb 14 22:54 logs
-rw-r--r-- 1 root root  1609 Feb 14 22:54 NOTICE
drwxr-xr-x 5 root root    79 Feb 14 22:56 plugins
-rw-r--r-- 1 root root  3070 Feb 14 22:54 README.rst

编辑配置

在三个节点上都创建数据库的数据存放位置
以manager节点为例

[root@manager crate-4.1.2]# mkdir -p /cratedb/data
[root@manager crate-4.1.2]# mkdir -p /cratedb/logs

进入数据区域目录下，将/opt目录（安装包解压后路径）下的crate.yml文件和log4j2.properties文件拷贝到数据区域目录（拷贝到非manager节点需要使用scp指令）。

[root@manager crate-4.1.2]# cd /cratedb/
[root@manager cratedb]# ll
total 0
drwxr-xr-x 2 root root 6 Jul 13 19:45 data
drwxr-xr-x 2 root root 6 Jul 13 19:45 logs
[root@manager config]# cp /opt/crate-4.1.2/config/* /cratedb/
## scp指令
## [root@manager config]# scp /opt/crate-4.1.2/config/* root@192.168.183.102:/cratedb/
## 
[root@manager config]# ll /cratedb/
total 32
-rw-r--r-- 1 root root 27305 Jul 13 19:49 crate.yml
drwxr-xr-x 2 root root     6 Jul 13 19:45 data
-rw-r--r-- 1 root root  2044 Jul 13 19:49 log4j2.properties
drwxr-xr-x 2 root root     6 Jul 13 19:45 logs
[root@manager config]# 

分别修改三个节点的crate.yml文件（/cratedb目录下的文件，启动时会指定配置文件）
以manager节点为例，从上至下
# Database Administration

auth.host_based.enabled: true

auth:
  host_based:
    config:
      0:
        user: crate
        address: _local_
        method: trust
      99:
        method: password

cluster.name: mycrate

node.name: "manager.learn.bigdata"

node.master: true

node.data: true

path.data: /cratedb/data

path.logs: /cratedb/logs

bootstrap.memory_lock: false

transport.tcp.port: 4300

http.port: 4200

gateway.recover_after_nodes: 2

gateway.expected_nodes: 3

discovery.seed_hosts:
  - manager.learn.bigdata:4300
  - master.learn.bigdata:4300
  - worker.learn.bigdata:4300

cluster.initial_master_nodes: ["manager.learn.bigdata", "master.learn.bigdata"]

以master为例，和manager不同的是域名，别的都一样。

node.name: "master.learn.bigdata"

启动服务

进入manager节点安装目录，使用下面指令启动服务，但是无反应

[root@manager crate-4.1.2]# ./bin/crate -d -Cpath.conf="/cratedb"

进入/cratedb/logs目录中查看日志

[root@manager cratedb]# cd logs
[root@manager logs]# ll
total 4
-rw-r--r-- 1 root root 2535 Jul 13 21:45 mycrate.log
[root@manager logs]# tail -1000f mycrate.log 
[2020-07-13T21:45:41,676][ERROR][o.e.b.BootstrapProxy     ] [manager.learn.bigdata] Exception
java.lang.RuntimeException: can not run crate as root
	at org.elasticsearch.bootstrap.BootstrapProxy.initializeNatives(BootstrapProxy.java:103) ~[crate-app.jar:4.1.2]
	at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:155) ~[crate-app.jar:4.1.2]
	at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:252) [crate-app.jar:4.1.2]
	at io.crate.bootstrap.CrateDB.init(CrateDB.java:158) [crate-app.jar:4.1.2]
	at io.crate.bootstrap.CrateDB.execute(CrateDB.java:138) [crate-app.jar:4.1.2]
	at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:82) [crate-app.jar:4.1.2]
	at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-7.0.0.jar:4.1.2]
	at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-7.0.0.jar:4.1.2]
	at io.crate.bootstrap.CrateDB.main(CrateDB.java:91) [crate-app.jar:4.1.2]
	at io.crate.bootstrap.CrateDB.main(CrateDB.java:84) [crate-app.jar:4.1.2]

说程序不能使用root用户运行，创建crate用户和组，并修改/cratedb子目录的所有者

#　增加组
[root@manager cratedb]# groupadd crate
# 增加用户
[root@manager cratedb]# useradd  -g crate crate
# 修改密码
[root@manager cratedb]# passwd crate
Changing password for user crate.
New password: 
BAD PASSWORD: The password is shorter than 8 characters
Retype new password: 
# 修改权限
[root@manager cratedb]# chown -R crate /cratedb/data/
[root@manager cratedb]# chown -R crate /cratedb/logs/

切换用户再执行

[crate@manager crate-4.1.2]$ ./bin/crate -d -Cpath.conf="/cratedb"
[crate@manager crate-4.1.2]$ Could not rename log file '/opt/crate-4.1.2/logs/gc.log' to '/opt/crate-4.1.2/logs/gc.log.00' (Permission denied).
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

不管怎么样，程序是运行起来了，查看4300和4200端口号

[root@manager crate-4.1.2]# netstat -antp | grep 4200
tcp6       0      0 192.168.183.101:4200    :::*                    LISTEN      9643/java           
[root@manager crate-4.1.2]# netstat -antp | grep 4300
tcp6       0      0 192.168.183.101:4300    :::*                    LISTEN      9643/java

浏览器访问，需要用户名和密码，考虑先安装crash。

上述要输入用户名密码的问题，我暂时先修改配置了，然后重启解决了。

auth.host_based.enabled: false

（安装crash）

我的环境里pip not found，所以我需要先安装epel-release
下载阿里的CentOS-Base源

[root@master java]# wget -O /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
--2020-07-14 13:25:40--  http://mirrors.aliyun.com/repo/Centos-7.repo
Resolving mirrors.aliyun.com (mirrors.aliyun.com)... 163.181.37.212, 163.181.37.205, 163.181.37.206, ...
Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|163.181.37.212|:80... failed: Connection refused.
Connecting to mirrors.aliyun.com (mirrors.aliyun.com)|163.181.37.205|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2523 (2.5K) [application/octet-stream]
Saving to: ‘/etc/yum.repos.d/CentOS-Base.repo’

100%[=======================================================================================================================================================================>] 2,523       --.-K/s   in 0s      

2020-07-14 13:26:08 (152 MB/s) - ‘/etc/yum.repos.d/CentOS-Base.repo’ saved [2523/2523]

[root@master java]# yum clean all
Loaded plugins: fastestmirror
Cleaning repos: base extras server updates
Cleaning up everything
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
Cleaning up list of fastest mirrors
[root@master yum.repos.d]# yum makecache
......

安装epel-release

[root@manager yum.repos.d]# yum install epel-release
[root@manager yum.repos.d]# yum install python-pip

查看pip版本

[root@manager yum.repos.d]# pip --version
pip 8.1.2 from /usr/lib/python2.7/site-packages (python 2.7)

安装

[root@manager yum.repos.d]# pip install crash
Collecting crash
......
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-nMRytq/Pygments/
You are using pip version 8.1.2, however version 20.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

需要升级pip

[root@manager yum.repos.d]# pip install --upgrade pip
Collecting pip
  Downloading https://files.pythonhosted.org/packages/43/84/23ed6a1796480a6f1a2d38f2802901d078266bda38388954d01d3f2e821d/pip-20.1.1-py2.py3-none-any.whl (1.5MB)
    100% |████████████████████████████████| 1.5MB 71kB/s 
Installing collected packages: pip
  Found existing installation: pip 8.1.2
    Uninstalling pip-8.1.2:
      Successfully uninstalled pip-8.1.2
Successfully installed pip-20.1.1

再次安装crash

[root@manager yum.repos.d]# pip install crash

[root@manager index]# crash --host=192.168.183.101:4200 -U crate
CONNECT OK
cr> 







USER: crate | HOSTS: 192.168.183.101:4200

其它节点加入

在其它节点启动crate服务，会自动加入集群当中，启动方法和manager节点一至。

基本操作

在UI界面命令行中操作

create table test (id int,name string,age int);
CREATE TABLE OK, 1 row affected (0.335 sec)

insert into test values(1,'zhangsan',20);
INSERT OK, 1 row affected (0.231 sec)

select * from test limit 100;
SELECT OK, 1 row in set (0.113 sec)
id	name	age
1 	 
zhangsan 	 
20

遇到的问题

问题一

[crate@manager crate-4.1.2]$ ./bin/crate -d -Cpath.conf="/cratedb"
[crate@manager crate-4.1.2]$ Unrecognized option: -Xlog:gc*,gc+age=trace,safepoint:file=/opt/crate-4.1.2/logs/gc.log:utctime,pid,tags:filecount=16,filesize=64m
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

原因为JDK版本问题，我的环境一开始装的是JDK1.8，出现问题后将JDK1.8升级到JDK12，再次启动的时候就好了。

问题二

[2020-07-13T22:51:06,507][ERROR][o.e.b.BootstrapProxy     ] [manager.learn.bigdata] node validation exception
[3] bootstrap checks failed
[1]: initial heap size [268435456] not equal to maximum heap size [478150656]; this can cause resize pauses and prevents mlockall from locking the entire heap
[2]: max file descriptors [4096] for crate process is too low, increase to at least [65536]
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] by adding `vm.max_map_count = 262144` to `/etc/sysctl.conf` or invoking `sysctl -w vm.max_map_count=262144`

命令启动后，服务启动失败，查看日志，出现上述错误，以下解决方法使用root用户。
解决[1]
增加环境变量vim /etc/profile，增加以下一句话，内存大小根据虚拟情况设置，设置完成后使用source /etc/profile生效
注意，修改完成后，使用的xshell客户端需要断开一次，再重新连接。

export CRATE_HEAP_SIZE=256M

解决[2]
编辑/etc/security/limits.conf文件

vim /etc/security/limits.conf
# 增加以下两句
crate   soft    nofile          65536
crate   hard    nofile          65536

修改后需退出crate用户再登录
解决[3]
编辑/etc/sysctl.conf文件

vim /etc/sysctl.conf
# 增加以下一句
vm.max_map_count = 262144
# 保存退出后执行以下命令
sysctl -w vm.max_map_count=262144