小弟今年研二,听师兄面试几乎全部问到分布式系统,实验室没有这方面的经验,几乎全挂。花了一天时间了解了下分布式系统,发现了中间件MyCat(国产)还是一款比较优秀的软件。实验室目前的项目普遍遇到了数据量较大的情况,使用分布式系统势在必行。话不多说,先在两台笔记本上安装了MySQL数据库,装完之后自带一个测试数据库world,其中包含三张表 city、country、countrylanguage。
部署步骤:
1.安装jdk,配置环境变量,不会请自行百度;
2.下载MyCat1.6 windows版本,链接在此:http://www.mycat.io/,解压后配置环境变量MYCAT_HOME:D:\soft_install\mycat
3.在解压后的路径下找到conf文件夹,配置server.xml、schema.xml。
server.xml内容如下所示:
<?xml version="1.0" encoding="UTF-8"?>
<!-- - - Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License. - You
may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0
- - Unless required by applicable law or agreed to in writing, software -
distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the
License for the specific language governing permissions and - limitations
under the License. -->
<!DOCTYPE mycat:server SYSTEM "server.dtd">
<mycat:server xmlns:mycat="http://org.opencloudb/">
<system>
<property name="defaultSqlParser">druidparser</property>
<!-- <property name="useCompression">1</property>--> <!--1为开启mysql压缩协议-->
<!-- <property name="processorBufferChunk">40960</property> -->
<!--
<property name="processors">1</property>
<property name="processorExecutor">32</property>
-->
<!--默认是65535 64K 用于sql解析时最大文本长度 -->
<!--<property name="maxStringLiteralLength">65535</property>-->
<!--<property name="sequnceHandlerType">0</property>-->
<!--<property name="backSocketNoDelay">1</property>-->
<!--<property name="frontSocketNoDelay">1</property>-->
<!--<property name="processorExecutor">16</property>-->
<!--
<property name="mutiNodeLimitType">1</property> 0:开启小数量级(默认) ;1:开启亿级数据排序
<property name="mutiNodePatchSize">100</property> 亿级数量排序批量
<property name="processors">32</property> <property name="processorExecutor">32</property>
<property name="serverPort">8066</property> <property name="managerPort">9066</property>
<property name="idleTimeout">300000</property> <property name="bindIp">0.0.0.0</property>
<property name="frontWriteQueueSize">4096</property> <property name="processors">32</property> -->
</system>
<user name="root">
<property name="password">123456</property>
<property name="schemas">robby</property>
</user>
<!--<user name="user">
<property name="password">user</property>
<property name="schemas">TESTDB</property>
<property name="readOnly">true</property>
</user>-->
<!-- <cluster> <node name="cobar1"> <property name="host">127.0.0.1</property>
<property name="weight">1</property> </node> </cluster> -->
<!-- <quarantine> <host name="1.2.3.4"> <property name="user">test</property>
</host> </quarantine> -->
</mycat:server>
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
schema.xml内容如下所示:
<?xml version="1.0"?>
<!DOCTYPE mycat:schema SYSTEM "schema.dtd">
<mycat:schema xmlns:mycat="http://org.opencloudb/">
<schema name="robby" checkSQLschema="false" sqlMaxLimit="100">
<table name="city" primaryKey="ID" type="global" dataNode="dn1,dn2" />
<table name="country" primaryKey="CODE" type="global" dataNode="dn1,dn2" />
</schema>
<dataNode name="dn1" dataHost="xps" database="world" />
<dataNode name="dn2" dataHost="sony" database="world" />
<dataHost name="xps" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<writeHost host="hostM1" url="localhost:3306" user="root"
password="123456">
</writeHost>
</dataHost>
<dataHost name="sony" maxCon="1000" minCon="10" balance="0"
writeType="0" dbType="mysql" dbDriver="native" switchType="1" slaveThreshold="100">
<heartbeat>select user()</heartbeat>
<!-- can have multi write hosts -->
<writeHost host="hostM1" url="223.3.87.165:3306" user="root"
password="123456">
</writeHost>
</dataHost>
<!--
<dataHost name="sequoiadb1" maxCon="1000" minCon="1" balance="0" dbType="sequoiadb" dbDriver="jdbc">
<heartbeat> </heartbeat>
<writeHost host="hostM1" url="sequoiadb://1426587161.dbaas.sequoialab.net:11920/SAMPLE" user="jifeng" password="jifeng"></writeHost>
</dataHost>
<dataHost name="oracle1" maxCon="1000" minCon="1" balance="0" writeType="0" dbType="oracle" dbDriver="jdbc"> <heartbeat>select 1 from dual</heartbeat>
<connectionInitSql>alter session set nls_date_format='yyyy-mm-dd hh24:mi:ss'</connectionInitSql>
<writeHost host="hostM1" url="jdbc:oracle:thin:@127.0.0.1:1521:nange" user="base" password="123456" > </writeHost> </dataHost>
<dataHost name="jdbchost" maxCon="1000" minCon="1" balance="0" writeType="0" dbType="mongodb" dbDriver="jdbc">
<heartbeat>select user()</heartbeat>
<writeHost host="hostM" url="mongodb://192.168.0.99/test" user="admin" password="123456" ></writeHost> </dataHost>
<dataHost name="sparksql" maxCon="1000" minCon="1" balance="0" dbType="spark" dbDriver="jdbc">
<heartbeat> </heartbeat>
<writeHost host="hostM1" url="jdbc:hive2://feng01:10000" user="jifeng" password="jifeng"></writeHost> </dataHost> -->
<!-- <dataHost name="jdbchost" maxCon="1000" minCon="10" balance="0" dbType="mysql"
dbDriver="jdbc"> <heartbeat>select user()</heartbeat> <writeHost host="hostM1"
url="jdbc:mysql://localhost:3306" user="root" password="123456"> </writeHost>
</dataHost> -->
</mycat:schema>
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
4.修改wrapper.conf文件,将其中的wrapper.java.command=java修改为wrapper.java.command=C:\Program Files (x86)\Java\jdk1.8.0_111\bin\java.exe
5.修改bin下面的startup_nowrap.bat的内容,主要是修改内存分配的大小
REM check JAVA_HOME & java
set "JAVA_CMD=%JAVA_HOME%/bin/java"
if "%JAVA_HOME%" == "" goto noJavaHome
if exist "%JAVA_HOME%\bin\java.exe" goto mainEntry
:noJavaHome
echo ---------------------------------------------------
echo WARN: JAVA_HOME environment variable is not set.
echo ---------------------------------------------------
set "JAVA_CMD=java"
:mainEntry
REM set HOME_DIR
set "CURR_DIR=%cd%"
cd ..
set "MYCAT_HOME=%cd%"
cd %CURR_DIR%
"%JAVA_CMD%" -server -Xms512M -Xmx1024M -XX:MaxPermSize=64M -XX:+AggressiveOpts -XX:MaxDirectMemorySize=1G -DMYCAT_HOME=%MYCAT_HOME% -cp "..\conf;..\lib\*" org.opencloudb.MycatStartup
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
6.以管理员身份打开cmd,然后切换到mycat\bin目录下,输入mycat install,然后输入mycat status查看是否安装成功,如果安装成功,则输入startup_nowrap,即可启动mycat服务;
7.然后使用mysql自带的workbench或者任意一个客户端,用户名为root,密码为123456,端口号为8066,默认数据库为robby,即可链接到mycat,然后进行操作。
几点体会:
1.mycat的主要目标就是管理多个数据库,将应用和数据库剥离,加了一层中间层,所以叫中间件;
2.mycat实现了读写分离,APP查询的时候可以查多台机器,写入的时候可以先写入一台机器,然后在合适的时机克隆到其他机器;
3.conf下面的rule.xml是很重要的配置,设置了很多表操作的规则,待深入学习;