无限水平扩容服务之 Infra

无限水平扩容服务之 Infra

概述

整洁架构 Infra 基础设施层，Dubbo 生产者服务角色，向注册中心注册服务，即可水平扩容。
但当 Dubbo 服务访问的数据库是 Primary/SECONDARY 架构时，Infra 需要进行改造。

改造调整

接上篇文章
1. 把H2切换成 Primary/SECONDARY 架构的 MySQL MGR集群
2. Spring Datasource Driver 切换成 ShardingSphere-JDBC

秒级搭建 MySQL MGR 集群

Docker和Docker-Compose环境下，使用 万里数据库 docker-compose yml
yml下载地址：https://gitee.com/GreatSQL/GreatSQL-Docker/blob/master/docker-compose/mgr-3nodes.yml

调整 mgr-3nodes.yml

调整了 镜像版本，ports，subnet

version: '2'

services:
  mgr2:
    image: greatsql/greatsql:8.0.25-16 #指定镜像
    container_name: mgr2    #设定容器名字
    hostname: mgr2          #设定容器中的主机名
    ports:
      - "3302:3306"
    networks:               #指定容器使用哪个专用网络
      mgr_net:
        ipv4_address: 172.16.0.2    #设置容器使用固定IP地址，避免重启后IP变化
    restart: unless-stopped         #设定重启策略
    environment:                    #设置多个环境变量
      TZ: Asia/Shanghai             #时区
      MYSQL_ROOT_PASSWORD: root                 #允许root账户空密码
      MYSQL_INIT_MGR: 1                             #初始化MGR集群
      MYSQL_MGR_LOCAL: '172.16.0.2:33061'           #当前MGR节点的local_address
      MYSQL_MGR_SEEDS: '172.16.0.2:33061,172.16.0.3:33061,172.16.0.4:33061'     #MGR集群seeds
      MYSQL_MGR_START_AS_PRIMARY: 1                 #指定当前MGR节点为Primary角色
      MYSQL_MGR_ARBITRATOR: 0
  mgr3:
    image: greatsql/greatsql:8.0.25-16
    container_name: mgr3
    hostname: mgr3
    ports:
      - "3303:3306"
    networks:
      mgr_net:
        ipv4_address: 172.16.0.3
    restart: unless-stopped
    depends_on:
      - "mgr2"
    environment:
      TZ: Asia/Shanghai
      MYSQL_ROOT_PASSWORD: root
      MYSQL_INIT_MGR: 1
      MYSQL_MGR_LOCAL: '172.16.0.3:33061'
      MYSQL_MGR_SEEDS: '172.16.0.2:33061,172.16.0.3:33061,172.16.0.4:33061'
      MYSQL_MGR_START_AS_PRIMARY: 0
      MYSQL_MGR_ARBITRATOR: 0                       #既非Primary，也非Arbitrator，那么就是Secondary角色了
  mgr4:
    image: greatsql/greatsql:8.0.25-16
    container_name: mgr4
    hostname: mgr4
    ports:
      - "3304:3306"
    networks:
      mgr_net:
        ipv4_address: 172.16.0.4
    restart: unless-stopped
    depends_on:
      - "mgr3"
    environment:
      TZ: Asia/Shanghai
      MYSQL_ROOT_PASSWORD: root
      MYSQL_INIT_MGR: 1
      MYSQL_MGR_LOCAL: '172.16.0.4:33061'
      MYSQL_MGR_SEEDS: '172.16.0.2:33061,172.16.0.3:33061,172.16.0.4:33061'
      MYSQL_MGR_START_AS_PRIMARY: 0
      MYSQL_MGR_ARBITRATOR: 0                   #指定当前MGR节点为Arbitrator角色，此时不能同时指定其为Primary/Secondary角色
networks:
  mgr_net:  #创建独立MGR专属网络
    ipam:
      config:
        - subnet: 172.16.0.0/24

创建并验证

创建
docker-compose -f mgr-3nodes.yml up -d

执行 SQL 验证
SELECT * FROM `performance_schema`.replication_group_members;

CHANNEL_NAME	MEMBER_ID	MEMBER_HOST	MEMBER_PORT	MEMBER_STATE	MEMBER_ROLE	MEMBER_VERSION
group_replication_applier	01a85e01-c945-11ed-96d3-0242ac100002	172.16.0.2	3306	ONLINE	PRIMARY	8.0.25
group_replication_applier	01cf9d21-c945-11ed-9646-0242ac100003	172.16.0.3	3306	ONLINE	SECONDARY	8.0.25
group_replication_applier	02121dfb-c945-11ed-9782-0242ac100004	172.16.0.4	3306	ONLINE	SECONDARY	8.0.25

Infra 切 ShardingSphere-JDBC

ssVersion 为 5.3.2 版本，集群模式向zookeeper注册，管理元数据

构建文件调整

implementation 'org.yaml:snakeyaml:1.33'
implementation "org.apache.shardingsphere:shardingsphere-cluster-mode-repository-zookeeper:${ssVersion}"
implementation "org.apache.shardingsphere:shardingsphere-jdbc-core:${ssVersion}"
runtimeOnly 'com.mysql:mysql-connector-j'

配置文件调整

server.port: 0
spring.application.name: demo-infra

---
spring.config.import:
  - classpath:/dubbo-service.yml

---
spring.datasource.driver-class-name: org.apache.shardingsphere.driver.ShardingSphereDriver
spring.datasource.url: jdbc:shardingsphere:classpath:ss-jdbc.yaml

---
spring.datasource.hikari:
  connection-timeout: 10000
  validation-timeout: 3000
  idle-timeout: 60000
  login-timeout: 5
  max-lifetime: 60000
  maximum-pool-size: 10
  minimum-idle: 5
  #       read-only: false
  auto-commit: false

---
spring.jpa:
  database-platform: MYSQL
  show-sql: true
  open-in-view: true
  generate-ddl: false
  hibernate:
    ddl-auto: none
    naming.physical-strategy: org.hibernate.boot.model.naming.CamelCaseToUnderscoresNamingStrategy
  properties.hibernate:
    dialect: org.hibernate.dialect.MySQL8Dialect
    format_sql: false
    use_sql_comments: true
    connection.autocommit: true

---
spring.sql.init:
  schema-locations: classpath:init-schema.sql
  mode: always # 启动后改为 never

ss-jdbc.yaml 集群模式 1主2从 读写分离 + 表分片

按领域分库后，水平分表

mode:
  type: Cluster
  repository:
    type: Zookeeper
    props:
      namespace: governance_ds
      server-lists: 192.168.1.168:2181
      retryIntervalMilliseconds: 500
      timeToLiveSeconds: 60
      maxRetries: 3
      operationTimeoutMilliseconds: 500

dataSources:
  write_account:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.cj.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.1.168:3302/account?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true&rewriteBatchedStatements=true
    username: root
    password: root
    maxPoolSize: 10
  read_account_1:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.cj.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.1.168:3303/account?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true
    username: root
    password: root
    maxPoolSize: 10
  read_account_2:
    dataSourceClassName: com.zaxxer.hikari.HikariDataSource
    driverClassName: com.mysql.cj.jdbc.Driver
    jdbcUrl: jdbc:mysql://192.168.1.168:3304/account?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8&allowPublicKeyRetrieval=true
    username: root
    password: root
    maxPoolSize: 10

rules:
  - !SHARDING
    tables:
      tbl_account:
        actualDataNodes: readwrite_ds.tbl_account_$->{0..1}
        databaseStrategy:
          none: # 分库策略 不分片
        tableStrategy:
          standard:
            shardingColumn: id
            shardingAlgorithmName: alg_mod
        keyGenerateStrategy:
          column: id
          keyGeneratorName: snowflake_generator
    bindingTables:
      - tbl_account

    # 分片算法配置
    shardingAlgorithms:
      alg_mod:
        type: MOD
        props:
          sharding-count: 2

    # 分布式序列算法配置
    keyGenerators:
      snowflake_generator:
        type: SNOWFLAKE

    # 读写分离配置
  - !READWRITE_SPLITTING
    dataSources:
      readwrite_ds: # 读写分离逻辑数据源名称
        staticStrategy:
          writeDataSourceName: write_account
          readDataSourceNames: [ read_account_1,read_account_2 ]
        loadBalancerName: round_robin
    loadBalancers:
      round_robin:
        type: ROUND_ROBIN

props:
  sql-show: true

init-schema.sql

注意主键id，不要设置自动增长

CREATE TABLE `tbl_account_0`
(
    `id`          bigint NOT NULL,
    `create_time` datetime(6) DEFAULT NULL,
    `update_time` datetime(6) DEFAULT NULL,
    `deleted`     bit(1) NOT NULL DEFAULT b'0' COMMENT '逻辑删除',
    `info`        json   NOT NULL COMMENT '信息',
    `mobile`      varchar(255)    DEFAULT NULL,
    `money`       bigint          DEFAULT NULL,
    `password`    varchar(255)    DEFAULT NULL,
    `user_name`   varchar(255)    DEFAULT NULL,
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_mobile` (`mobile`),
    UNIQUE KEY `uk_user_name` (`user_name`),
    KEY           `idx_deleted` (`deleted`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;

CREATE TABLE `tbl_account_1`
(
    `id`          bigint NOT NULL,
    `create_time` datetime(6) DEFAULT NULL,
    `update_time` datetime(6) DEFAULT NULL,
    `deleted`     bit(1) NOT NULL DEFAULT b'0' COMMENT '逻辑删除',
    `info`        json   NOT NULL COMMENT '信息',
    `mobile`      varchar(255)    DEFAULT NULL,
    `money`       bigint          DEFAULT NULL,
    `password`    varchar(255)    DEFAULT NULL,
    `user_name`   varchar(255)    DEFAULT NULL,
    PRIMARY KEY (`id`),
    UNIQUE KEY `uk_mobile` (`mobile`),
    UNIQUE KEY `uk_user_name` (`user_name`),
    KEY           `idx_deleted` (`deleted`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci;

主键调整

使用 ShardingSphere keyGeneratorName 生成主键，领域模型调整为

@Id
@Column(name = Fields.id)
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;

多实例 Infra

经上述调整，数据库为 MGR 集群，Infra 也支持水平扩容了，本地调试开发时，
若用 IntelliJ IDEA，可设置 Allow multiple instances 模式，启动多个 Infra 实例，
登录 Nacos，服务管理->服务列表 可看到 服务名对应的实例数目。