Canal-保存mysql篇

wanping15825992341

于 2022-08-10 18:25:12 发布

阅读量386

点赞数

文章标签： mysql java 数据库

本文链接：https://blog.csdn.net/wanping15825992341/article/details/126272038

版权

Canal-保存mysql篇

一、java实现

先用java代码手写一遍，方便后续业务逻辑理解

1、maven配置：

<dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <!--mysql-->
        <dependency>
            <groupId>mysql</groupId>
            <artifactId>mysql-connector-java</artifactId>
            <version>8.0.24</version>
        </dependency>

        <dependency>
            <groupId>commons-dbutils</groupId>
            <artifactId>commons-dbutils</artifactId>
            <version>1.6</version>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-jdbc</artifactId>
        </dependency>

        <dependency>
            <groupId>com.alibaba.otter</groupId>
            <artifactId>canal.client</artifactId>
            <version>1.1.4</version>
        </dependency>

2、application.properties配置

# 服务端口
server.port=10000
# 服务名
spring.application.name=canal-client

# 环境设置：dev、test、prod
spring.profiles.active=dev

# mysql数据库连接
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.datasource.url=jdbc:mysql://localhost:3306/cancal_mysql?serverTimezone=GMT%2B8
spring.datasource.username=root
spring.datasource.password=603409875

3、代码实现

首先声明一个队列来接收sql语句

		//sql队列
    private Queue<String> SQL_QUEUE = new ConcurrentLinkedQueue<>();
		@Resource
    private DataSource dataSource;

编写主流程方法

// 创建链接
        CanalConnector connector = CanalConnectors.newSingleConnector(
                new InetSocketAddress("127.0.0.1",
                11111), "example", "", "");
        int batchSize = 1000;
        try {
     
            connector.connect();
            connector.subscribe("cancal_mysql\\..*");//订阅cancal_mysql库下的全表
            connector.rollback();
            try {
     
                while (true) {
     
                    //尝试从master那边拉去数据batchSize条记录，有多少取多少
                    Message message = connector.getWithoutAck(batchSize);
                    long batchId = message.getId();
                    int size = message.getEntries().size();
                    if (batchId == -1 || size == 0) {
     
                        Thread.sleep(1000);
                    } else {
     
                        //数据处理
                        dataHandle(message.getEntries());
                    }
                    connector.ack