DCache >> DCache测试搭建

DCache测试搭建

简介

背景

Dcache使用腾讯Tars框架开发，属于分布式的NoSQL存储系统。数据存储在内存中，还可以连接后端DB做数据的持久化。

支持平台

Linux

特点

• 高性能存储引擎

  • 支持多种数据结构，如：
    • key-value
    • k-k-row
    • list
    • set
    • zset
  • 支持数据持久化到后端DB

• 集群模式

  • 高可用
  • 高扩展
  • 异地镜像
  • 就近接入

• 通过名字访问

  • 支持同步、异步、单向RPC调用

• 高效运维平台(提供WebUI)

  • 服务部署
  • 扩缩容
  • 迁移
  • 服务配置
  • 质量监控

组成

• tars框架
• 业务服务
  • KVCache模块
    • 存储key-value
  • MKVcache模块
    • k-k-row
    • list
    • set
    • zset
• 公共基础服务
  • OptServer
  • ConfigServer
  • PropertyServer
  • DCache管理平台

安装部署

tars框架

依赖环境

1. 操作系统

   • linux内核：2.6.18 or later

2. C++语言框架依赖

   • gcc：4.8.2 or later
   • glibc-devel
   • bison：2.5 or later
   • flex：2.5 or later
   • cmake：2.8.8 or later
   • rapidjson：1.0.2

3. tars框架运行依赖

   • mysql：5.6.5 or later

4. web管理系统依赖

   • nvm：0.33.11 or later
   • node：8.11.3 or later

依赖安装

$ yum -y install gcc gcc-c++ cmake yasm glibc-devel flex bison ncurses-devel zlib-devel autoconf python-requests

mysql部署

1. 下载mysql源码包(5.6.44版本), 官方要求数据库版本不低于5.6.5。

$ wget "https://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.44.tar.gz"

2. 安装 - 编译安装

# 创建普通用户
$ useradd -m -d /home/mysql mysql

# 解压源码包
$ tar zxf mysql-5.6.44.tar.gz

# 创建mysql安装目录, 并做软连接
$ mkdir /usr/local/mysql-5.6.44
$ ln -s /usr/local/mysql-5.6.44 /usr/local/mysql

# 修改属主属组
$ chown mysql:mysql /usr/local/mysql-5.6.44 /usr/local/mysql

# 切换到源码目录，编译安装
$ cd mysql-5.6.44
$ cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mysql-5.6.44 -DWITH_INNOBASE_STORAGE_ENGINE=1 -DMYSQL_USER=mysql -DDEFAULT_CHARSET=utf8 -DDEFAULT_COLLATION=utf8_general_ci
$ make && make install

3. mysql服务配置

# 删除原始数据目录, 在数据盘创建数据目录，进行软连
$ rm -rf /usr/local/mysql/data
$ mkdir -p /data/mysql-data
$ ln -s /data/mysql-data /usr/local/mysql/data

# 修改属主属组
$ chown -R mysql:mysql /data/mysql-data /usr/local/mysql/data

# 配置启动脚本
$ cd /usr/loca/mysql/
$ cp support-files/mysql.server /etc/init.d/mysql

# 初始化数据库
$ rm -rf /etc/my.cnf
$ yum -y install perl
$ yum install -y perl-Module-Install.noarch
$ perl scripts/mysql_install_db --user=mysql --datadir=/usr/local/mysql/data --basedir=/usr/local/mysql --pid-file=/var/run/mysql/mysql.pid --socket=/tmp/mysql.sock 

# 修改配置文件
$ vim /usr/local/mysql/my.cnf
'''
[mysqld]

# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
innodb_buffer_pool_size = 128M

# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
log_bin

# These are commonly set, remove the # and set as required.
basedir = /usr/local/mysql
datadir = /usr/local/mysql/data
# port = .....
# server_id = .....
socket = /tmp/mysql.sock

bind-address={$your machine ip}

# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
join_buffer_size = 128M
sort_buffer_size = 2M
read_rnd_buffer_size = 2M

sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
'''

# 启动服务，设置开机自启
$ service mysql start
$ chkconfig mysql on

4. 设置环境变量

$ echo -e 'PATH=$PATH:/usr/local/mysql/bin\nexport PATH' > /etc/profile.d/mysql_env.sh
$ source /etc/profile

5. 修改root初始密码, 授权登录

$ mysqladmin -u root password 'root@appinside'
$ mysql -uroot -p
mysql> grant all on *.* to "root"@"%" identified by 'root@appinside' with grant option;
mysql> grant all on *.* to "root"@"localhost" identified by 'root@appinside' with grant option;
mysql> grant all on *.* to "root"@"10.80.137.193" identified by 'root@appinside' with grant option;
mysql> grant all on *.* to "root"@"platformxxxx.cdn.idc.com" identified by 'root@appinside' with grant option;

tars 内部程序通过’root@{主机名}'登录数据库，需要授权。主机名可以通过/etc/hosts查看。

6. 添加mysql库路径

$ vim /etc/ld.so.conf

/usr/local/mysql/lib/
ldconfig

7. mysql主从配置

测试环境暂不需要，待正式部署后再做补充。#TODO

tars部署

1. git clone Tars

# 使用递归方式clone
$ git clone  https://github.com/TarsCloud/Tars.git --recursive

2. 使用一键部署脚本

$ cd Tars/de