基于MP提供的dynamic-datasource实现多数据源

xujinwei_gingko

于 2024-10-11 16:00:20 发布

阅读量186

点赞数 1

分类专栏： JAVA 文章标签： mybatis 数据库

本文链接：https://blog.csdn.net/xujinwei_gingko/article/details/142854655

版权

JAVA 专栏收录该内容

43 篇文章 0 订阅

订阅专栏

随着项目规模的扩大，单一数据源已无法满足复杂业务需求，多数据源（动态数据源）应运而生。项目遇到的多数据源场景：通过Flink CDC采集某个数据源变化数据（主表数据），变化数据（主表数据） + 源数据源的变化数据的辅表数据 + 映射转换程序计算得到最终结果数据，并将最终的结果落库保存在另外一个数据源中，上述的需求中就需要2个数据源，源数据源查询数据，目标数据源更新数据。

本文基于MP提供的dynamic-datasource实现多数据源，springboot应用，源数据源负责查询数据，不需要事务支持，目标数据源需要更新数据，需要数据库事务，实现步骤如下：

1、增加依赖 pom.xml

核心依赖：

baomidou的dynamic-datasource 动态数据源依赖
HikariCP 数据源的连接池
Mybatis plus

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.7.6</version>
        <relativePath/>
    </parent>

    <groupId>com.gingko</groupId>
    <artifactId>springboot</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>springboot</name>

    <dependencies>
        <!-- 依赖spring-web-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <!-- springboot默认数据源（HikariCP)-->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>
        <!-- mysql驱动-->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.16</version>
        </dependency>
        <!-- 整合MyBatis -->
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>mybatis-plus-boot-starter</artifactId>
           <version>3.4.2</version>
        </dependency>
        <!-- SpringBoot 的AOP实现 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-aop</artifactId>
        </dependency>
        <!-- junit 依赖 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
        </dependency>
        <!-- SpringBoot 健康检查 -->
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>
        <!-- lombok -->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.0</version>
            <scope>provided</scope>
        </dependency>
        <!-- jwt token -->
        <dependency>
            <groupId>io.jsonwebtoken</groupId>
            <artifactId>jjwt-api</artifactId>
            <version>0.10.7</version>
        </dependency>
        <dependency>
            <groupId>io.jsonwebtoken</groupId>
            <artifactId>jjwt-impl</artifactId>
            <version>0.10.7</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.jsonwebtoken</groupId>
            <artifactId>jjwt-jackson</artifactId>
            <version>0.10.7</version>
            <scope>runtime</scope>
        </dependency>
        <!-- fastjson -->
        <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version>1.2.83</version>
        </dependency>
        <!--kafka -->
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>
        <!--dynamic-datasource -->
        <dependency>
            <groupId>com.baomidou</groupId>
            <artifactId>dynamic-datasource-spring-boot-starter</artifactId>
            <version>3.5.2</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>${maven.compiler.source}</source>
                    <target>${maven.compiler.target}</target>
                    <encoding>${project.build.sourceEncoding}</encoding>
                </configuration>
            </plugin>
        </plugins>
    </build>

</project>

2、配置多数据源 application.yml

配置了2个数据源，master数据源（目标数据源，负责计算结果的写入），slave数据源（源数据源，负责源数据的查询）

3、2个数据源的服务类

UserServiceImpl是源数据源@DS("slave")服务类，查询数据不需要事务支持

StudentServiceImpl是目标数据源@DS("master")服务类，更新数据需要事务支持

TestServiceImpl的逻辑模拟了2个数据源的操作，源数据源读取数据，目标数据源更新数据

package com.gingko.service.impl;

import com.baomidou.dynamic.datasource.annotation.DS;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.gingko.entity.User;
import com.gingko.exception.BusinessException;
import com.gingko.mapper.UserMapper;
import com.gingko.service.UserService;
import org.springframework.stereotype.Service;

import javax.annotation.Resource;

/**
 * slave数据库的需求都是查询，无需事务支持
 */
@Service
public class UserServiceImpl implements UserService {

    @Resource
    private UserMapper userMapper;
    @DS("slave")
    @Override
    public User getByUserId(String userId) throws BusinessException {
        QueryWrapper<User> queryWrapper = new QueryWrapper<>();
        queryWrapper.eq("user_id",userId);
        User user = this.userMapper.selectOne(queryWrapper);
        return user;
    }
}

package com.gingko.service.impl;

import com.baomidou.dynamic.datasource.annotation.DS;
import com.baomidou.mybatisplus.core.conditions.query.QueryWrapper;
import com.baomidou.mybatisplus.core.metadata.IPage;
import com.baomidou.mybatisplus.extension.plugins.pagination.Page;
import com.gingko.dto.StudentDto;
import com.gingko.entity.Student;
import com.gingko.exception.BusinessException;
import com.gingko.exception.BusinessExceptionCode;
import com.gingko.mapper.StudentMapper;
import com.gingko.service.StudentService;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;

import javax.annotation.Resource;

@Service
public class StudentServiceImpl implements StudentService {
    @Resource
    private StudentMapper studentMapper;

    /**
     * 从slave读取数据，加工后，维护到master数据库，master数据源需要事务
     * @throws BusinessException
     */
    @Transactional(rollbackFor = Exception.class)
    @DS("master")
    @Override
    public void batchInsert() throws BusinessException {
        for(int i=10;i<20;i++) {
            if(i==13) {
                throw new BusinessException(BusinessExceptionCode.SIMULATE_EXCEPTION);
            }
            Student student = new Student(i+"","张三"+i,i,"1");
            studentMapper.insert(student);
        }
    }

    @Transactional(rollbackFor = Exception.class)
    @DS("master")
    @Override
    public IPage<Student> getByPage(long pageNum, long pageSize) throws BusinessException {
        IPage<Student> page = new Page<>(pageNum,pageSize);
        QueryWrapper<Student> queryWrapper = new QueryWrapper<>();
        //queryWrapper.gt("age",10);
        page = this.studentMapper.selectPage(page, queryWrapper);
        return page;
    }

    @Transactional(rollbackFor = Exception.class)
    @DS("master")
    @Override
    public IPage<StudentDto> getStudentInfoByPage(long pageNum, long pageSize, int minAge) throws BusinessException {
        IPage<StudentDto> page = new Page<>(pageNum,pageSize);
        page = this.studentMapper.selectStudentMap(page,minAge);
        return page;
    }

    public StudentMapper getStudentMapper() {
        return studentMapper;
    }

    public void setStudentMapper(StudentMapper studentMapper) {
        this.studentMapper = studentMapper;
    }
}

package com.gingko.service.impl;

import com.gingko.entity.User;
import com.gingko.service.StudentService;
import com.gingko.service.TestService;
import com.gingko.service.UserService;
import org.springframework.stereotype.Service;

import javax.annotation.Resource;

@Service
public class TestServiceImpl implements TestService {

    @Resource
    private StudentService studentService;
    @Resource
    private UserService userService;
    @Override
    public void simulate2Datasource() {
        //模拟从slave数据库读取数据
        User thunder = userService.getByUserId("thunder");
        //模拟到master数据库写入数据
        studentService.batchInsert();
    }
}

4、测试类：StudentController

模拟发送请求，同时操作2个数据源

package com.gingko.controller;

import com.baomidou.mybatisplus.core.metadata.IPage;
import com.gingko.common.GenericWebResult;
import com.gingko.dto.StudentDto;
import com.gingko.service.StudentService;
import com.gingko.service.TestService;
import org.springframework.web.bind.annotation.*;

import javax.annotation.Resource;

@RestController
@RequestMapping("student")
public class StudentController {

    @Resource
    private StudentService studentService;
    @Resource
    private TestService testService;

    @PostMapping("/simulate")
    public GenericWebResult simulate() {
        testService.simulate2Datasource();
        return GenericWebResult.ok("从slave数据源读取数据，经过转换程序后，写入master数据源成功");
    }

        
}

5、运行结果及分析：

项目启动后，2个数据源及对应的连接池都已准备好