canal和hbase自带protobuf版本冲突问题

最新推荐文章于 2022-11-30 14:29:54 发布

天河~

最新推荐文章于 2022-11-30 14:29:54 发布

阅读量805

点赞数

本文链接：https://blog.csdn.net/xinguopeng/article/details/103054600

版权

java 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

测试使用canal解析mysql binlog并把数据插入hbase的时候程序出现异常，测试过程如下：
测试中使用软件的版本分别为：

canal：canal.deployer-1.1.1
hbase：1.2.0-cdh5.16.1

pom.xml配置信息如下：

<dependencies>
	<dependency>
	    <groupId>org.apache.hbase</groupId>
	    <artifactId>hbase-client</artifactId>
	    <version>1.2.0</version>
	</dependency>

    <dependency>
		<groupId>com.alibaba.otter</groupId>
		<artifactId>canal.client</artifactId>
		<version>1.1.1</version>
	</dependency>
</dependencies>

程序运行的现象为，hbase可以正常操作，使用canal功能时报异常，信息如下：

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/D:/Program%20Files/maven_local_repository/org/slf4j/slf4j-log4j12/1.6.1/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/D:/Program%20Files/maven_local_repository/ch/qos/logback/logback-classic/1.1.3/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
Exception in thread "Thread-0" java.lang.NoClassDefFoundError: com/google/protobuf/GeneratedMessageV3
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.doConnect(SimpleCanalConnector.java:151)
	at com.alibaba.otter.canal.client.impl.SimpleCanalConnector.connect(SimpleCanalConnector.java:113)
	at com.cn.call_end.canal.Client_Admin_User_Attr.run(Client_Admin_User_Attr.java:73)
Caused by: java.lang.ClassNotFoundException: com.google.protobuf.GeneratedMessageV3
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 15 more

查了一下原因是hbase使用的protobuf（hbase使用2.5版本）版本与canal使用的protobuf版本冲突，解决方法如下
修改pom.xml

<dependencies>
	<dependency>
    	<groupId>org.apache.hbase</groupId>
        <artifactId>hbase-shaded-client</artifactId>
        <version>1.2.0</version>
    </dependency>

    <dependency>
        <groupId>org.apache.hbase</groupId>
        <artifactId>hbase-shaded-server</artifactId>
        <version>1.2.0</version>
    </dependency>

    <dependency>
		<groupId>com.alibaba.otter</groupId>
		<artifactId>canal.client</artifactId>
		<version>1.1.1</version>
	</dependency>
	
	<!-- https://mvnrepository.com/artifact/com.google.protobuf/protobuf-java -->
	<dependency>
	    <groupId>com.google.protobuf</groupId>
	    <artifactId>protobuf-java</artifactId>
	    <version>3.5.1</version>
	</dependency>
</dependencies>

再次运行程序，问题解决