java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString

最新推荐文章于 2023-05-11 00:43:08 发布
原创 最新推荐文章于 2023-05-11 00:43:08 发布 · 3w 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#hadoop #spark #hbase #bug

本文详细介绍了Hadoop和Spark在操作HBase时遇到的Java.lang. IllegalAccessError问题的根本原因，并提供了在YARN和Spark环境下解决此问题的具体步骤和配置方法。

hadoop mr 或者 spark 操作 hbase时候就出现这个错误
这是hbase的bug,可在jira上看到该问题:https://issues.apache.org/jira/browse/HBASE-10304

报错信息:

15/08/17 19:28:33 ERROR yarn.ApplicationMaster: User class threw exception: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString ca 
nnot access its superclass com.google.protobuf.LiteralByteString 
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString 
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:210) 
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:121) 
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90) 
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264) 
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:169) 
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:164) 
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:107) 
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:736) 
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:178) 
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:82) 
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isTableAvailable(HConnectionManager.java:962) 
        at org.apache.hadoop.hbase.client.HBaseAdmin.isTableAvailable(HBaseAdmin.java:1081) 
        at org.apache.hadoop.hbase.client.HBaseAdmin.isTableAvailable(HBaseAdmin.java:1089) 
        at com.umeng.dp.yuliang.play.HBaseToES$.main(HBaseToES.scala:28) 
        at com.umeng.dp.yuliang.play.HBaseToES.main(HBaseToES.scala) 
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
        at java.lang.reflect.Method.invoke(Method.java:606) 
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:483) 
Caused by: java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString 
        at java.lang.ClassLoader.defineClass1(Native Method) 
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800) 
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) 
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71) 
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361) 
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
        at java.security.AccessController.doPrivileged(Native Method) 
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425) 
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:930) 
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildScanRequest(RequestConverter.java:434) 
        at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:297) 
        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:157) 
        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:57) 
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114) 
        ... 18 more

hadoop yarn 解决方案:

  1. 提交作业方式

    $ export HADOOP_CLASSPATH="/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar"
$ ./hadoop-2.2.0/bin/hadoop --config /home/stack/conf_hadoop/ jar ./hbase/hbase-assembly/target/hbase-0.99.0-SNAPSHOT-job.jar  org.apache.hadoop.hbase.mapreduce.RowCounter usertable

  2. 增加HADOOP_CLASSPATH到linux环境变量中
    增加如下内容到bashrc 或者 bash_profile 或者 profile ,这样是linux环境变量中就行
    export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar

spark 解决方案:

  1. 提交作业方式
    -conf 增加spark.driver.extraClassPath & spark.executor.extraClassPath

    spark-submit --class com.umeng.dp.yuliang.play.HBaseToES --master yarn-cluster --conf "spark.driver.extraClassPath=/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar" --conf "spark.executor.extraClassPath=/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar"   --jars /home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar ScalaMR-0.0.1-jar-with-dependencies.jar

  2. 增加如下配置到$SPARK_HOME/conf/spark-defaults.conf文件

    spark.driver.extraClassPath /home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar
spark.executor.extraClassPath /home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar

spark maillist:
http://apache-spark-user-list.1001560.n3.nabble.com/java-lang-IllegalAccessError-class-com-google-protobuf-HBaseZeroCopyByteString-cannot-access-its-supg-tc24303.html
尊重原创，未经允许不得转载：
http://blog.csdn.net/stark_summer/article/details/47750017

HBASE: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>
飞流
08-31 3万+
hbase查询时，出现以下异常： Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.()V from class org.apache.hadoop.hbase
java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
q79969786的专栏
08-30 3419
14/08/30 19:52:37 INFO zookeeper.ClientCnxn: Socket connection established to n3.example.com/192.168.137.103:2181, initiating session 14/08/30 19:52:37 INFO zookeeper.ClientCnxn: Session establishmen
IllegalAccessError HBaseZeroCopyByteString
04-07 173
obuf.LiteralByteString  这个是protobuf版本不一致造成的，但是缺少hbase jar也会报错
java.lang.IllegalAccessError: class javax.activation.SecuritySupport12 cannot access its superclass
runfeel
07-12 752
最近加入新的项目组，eclipse + tomcat7 + spring +ibatis + restful 遇到了这样的问题， 说是不能访问父类，我一开始以为是版本的原因，但是久经更改，错误依然，实在累了， 最终的解决办法是我把SecuritySupport12 所在的jar包activation.jar 删除了，这样在tomcat就可以发布了， 但是这终究不是很好的解决办法，也只能先暂时...
java.long.NoClassDefFoundError:com/google/protobuf/LiteralByteString
mmj123321123的博客
03-25 1492
flink实时数据仓库dwd层从kafka中拿数据，经过FlinkAPP消费，维度数据存放到hbase，事实表存到kafka
"org.springframework.beans.factory.BeanCreationException","error.message":"Error creating bean with name 'withDrawRewardImpl': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cn.migu.wheat.consumer.ConsumerProxy#23': Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.lang.Object]: Factory method 'getProxyInstance' threw exception; nested exception is java.lang.IllegalAccessError: class cn.migu.reading.server.readduration.ImportDurationBlackListCptsRequest tried to access method 'com.google.protobuf.LazyStringArrayList com.google.protobuf.LazyStringArrayList.emptyList()' (cn.migu.reading.server.readduration.ImportDurationBlackListCptsRequest and com.google.protobuf.LazyStringArrayList are in unnamed module of loader 'app')", 翻译成中文
06-18
###解决方案步骤1.**升级Protobuf版本**：在`pom.xml`中，将protobuf-java升级到最新稳定版（如3.22.2）：```xml<dependency><groupId>com.google.protobuf</groupId><artifactId>protobuf-java</artifactId>...
6月 25, 2025 9:26:40 下午 com.google.testing.platform.main.MainKt$main$2 uncaughtException 严重: Fatal error while executing main with args: --proto_config=C:\Users\bowen.xiao\.android\utp\runnerConfig12320517331293124685.pb --proto_server_config=C:\Users\bowen.xiao\.android\utp\serverConfig104787935992664716.pb java.lang.IllegalAccessError: class com.google.protobuf.GeneratedMessageV3 tried to access method 'boolean com.google.protobuf.CodedInputStream.shouldDiscardUnknownFields()' (com.google.protobuf.GeneratedMessageV3 is in unnamed module of loader java.net.URLClassLoader @2102a4d5; com.google.protobuf.CodedInputStream is in unnamed module of loader 'app') at com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:325) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry.<init>(AppProcessesProto.java:119) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry.<init>(AppProcessesProto.java:54) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry$1.parsePartialFrom(AppProcessesProto.java:770) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry$1.parsePartialFrom(AppProcessesProto.java:764) at com.google.protobuf.CodedInputStream$ArrayDecoder.readMessage(CodedInputStream.java:980) at com.android.server.adb.protos.AppProcessesProto$AppProcesses.<init>(AppProcessesProto.java:871) at com.android.server.adb.protos.AppProcessesProto$AppProcesses.<init>(AppProcessesProto.java:821) at com.android.server.adb.protos.AppProcessesProto$AppProcesses$1.parsePartialFrom(AppProcessesProto.java:1558) at com.android.server.adb.protos.AppProcessesProto$AppProcesses$1.parsePartialFrom(AppProcessesProto.java:1552) at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:111) at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:126) at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:20) at com.android.server.adb.protos.AppProcessesProto$AppProcesses.parseFrom(AppProcessesProto.java:1014) at com.android.ddmlib.internal.DeviceClientMonitorTask$TrackAppProcessor.processIncomingData(DeviceClientMonitorTask.java:498) at com.android.ddmlib.internal.DeviceClientMonitorTask.run(DeviceClientMonitorTask.java:245) at java.base/java.lang.Thread.run(Thread.java:842) 6月 25, 2025 9:26:40 下午 com.google.testing.platform.RunnerImpl stop 严重: Termination signal received. Stopping test execution. 6月 25, 2025 9:26:40 下午 com.google.testing.platform.runtime.android.driver.AndroidInstrumentationDriver cancel 警告: Cancellation triggered in Android Instrumentation Driver. 6月 25, 2025 9:26:40 下午 com.google.testing.platform.RunnerImpl stop 警告: Waiting for UTP runner to exit... 6月 25, 2025 9:26:40 下午 com.google.testing.platform.RunnerImpl stop 严重: Aborted execution. Interim results: test_status: ABORTED
06-26
我们面对的是一个Android测试执行过程中的IllegalAccessError异常，具体错误信息是："IllegalAccessErrorcom.google.protobuf.GeneratedMessageV3accessmethodCodedInputStreamshouldDiscardUnknownFields"这个错误...
> Task :scripts:connectedDebugAndroidTest 4�� 28, 2025 7:23:36 ���� com.google.testing.platform.main.MainKt$main$2 uncaughtException ����: Fatal error while executing main with args: --proto_config=C:\Users\bowen.xiao\.android\utp\runnerConfig13203883161098104307.pb --proto_server_config=C:\Users\bowen.xiao\.android\utp\serverConfig14046213769878603943.pb java.lang.IllegalAccessError: class com.google.protobuf.GeneratedMessageV3 tried to access method 'boolean com.google.protobuf.CodedInputStream.shouldDiscardUnknownFields()' (com.google.protobuf.GeneratedMessageV3 is in unnamed module of loader java.net.URLClassLoader @77b7ffa4; com.google.protobuf.CodedInputStream is in unnamed module of loader 'app') at com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:325) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry.<init>(AppProcessesProto.java:119) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry.<init>(AppProcessesPro
04-29
好的，我现在需要帮助用户解决在执行Task :scripts:connectedDebugAndroidTest时出现的java.lang.IllegalAccessError错误，特别是涉及com.google.protobuf.GeneratedMessageV3和CodedInputStream ...
11239 [Thread-9] ERROR org.apache.hadoop.hive.ql.exec.TaskRunner - Error in executeTask java.lang.IllegalAccessError: tried to access field com.google.protobuf.AbstractMessage.memoizedSize from class org.apache.tez.dag.api.records.DAGProtos$ConfigurationProto at org.apache.tez.dag.api.records.DAGProtos$ConfigurationProto.getSerializedSize(DAGProtos.java:21080) at com.google.protobuf.AbstractMessageLite.writeTo(AbstractMessageLite.java:75) at org.apache.tez.client.TezClientUtils.createApplicationSubmissionContext(TezClientUtils.java:571) at org.apache.tez.client.TezClient.setupApplicationContext(TezClient.java:500) at org.apache.tez.client.TezClient.start(TezClient.java:400) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.startSessionAndContainers(TezSessionState.java:433) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.openInternal(TezSessionState.java:368) at org.apache.hadoop.hive.ql.exec.tez.TezSessionPoolSession.openInternal(TezSessionPoolSession.java:124) at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.open(TezSessionState.java:245) at org.apache.hadoop.hive.ql.exec.tez.TezTask.ensureSessionHasResources(TezTask.java:368) at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:195) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:205) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:97) at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:76) FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.tez.TezTask. tried to access field com.google.protobuf.AbstractMessage.memoizedSize from class org.apache.tez.dag.api.records.DAGProtos$ConfigurationProto 12592 [36b06449-3943-49a2-935e-f4ecfa74aa67 main] ERROR org.apache.hadoop.hive.ql.Driver - FAILED: Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.tez.TezTask. tried to access field com.google.protobuf.AbstractMessage.memoizedSize from class org.apache.tez.dag.api.records.DAGProtos$ConfigurationProto
最新发布
09-27
`java.lang.IllegalAccessError: tried to access field com.google.protobuf.AbstractMessage.memoizedSize` 错误通常是由于 Protobuf 版本不兼容导致的。当不同的库依赖于不同版本的 Protobuf 时，就可能出现这种...
nested exception is java.lang.IllegalAccessError: tried to access method net.sf.ehcache.CacheManager.()V from class org.hibernat
大树下那片阴凉
06-16 5808
今天配置struts2+hibernate3+spring2时出现这种问题,控制台输出了java.lang.IllegalAccessError: tried to access method net.sf.ehcache.CacheManager.()V from class org.hibernate.cache.EhCacheProvider上网查了一下.原来是JAR文件版本问题 
java报错--＞java.lang.IllegalAccessError
Hitvz的博客
05-11 3865
在gradle中运行main方法突然出现如下错误看报错是资源访问不了，因为用了jdk9之后的jdk版本，会有访问问题，但是我写main方法又不是模块，知道要加jvm参数，但我用的gradle，就很迷茫，怎么设置参数。
JAVA错误解决：tomcat启动错误（java.lang.IllegalAccessError: class javax.xml.parsers.SecuritySupport12 cannot ）
wwpp1的博客
11-21 3573
解决方案：在web.xml中增加信息：metadata-complete="true"  错误如下： java.lang.IllegalAccessError: class javax.xml.parsers.SecuritySupport12 cannot access its superclass javax.xml.parsers.SecuritySupport at java.l...
IllegalAccessError
teasp的专栏
05-17 947
    今天在工作中遇到了这样一个Error，IllegalAccessError。java.lang.IllegalAccessError: class sun.reflect.GeneratedConstructorAccessor3 cannot access its superclass sun.reflect.ConstructorAccessorImpl。本系统使用了OSGI，初步分析...
ZeroCopyLiteralByteString cannot access superclass
宏轩ＪＡＶＡ技术专栏
04-23 2409
问题描述 在HBase上运行MapReduce作业时，报如下异常：IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString 使用HBase环境如下：CDH5.0.1, HB
记一次，继承出现的问题“cannot access its superclass”
大鲨鱼丶
06-08 2001
记一次，spring-boot-devtools引发的类加载器问题
java.lang.NoClassDefFoundError:Could not initialize class org.apache.hadoop.hbase.util.ByteStringer
一杯派蒙的博客
09-22 2075
java.lang.NoClassDefFoundError: Could not initialize class org.apache.hadoop.hbase.util.ByteStringer 版本冲突。 protobuf-java 需要是3.0.0 以下的。 <dependency> <groupId>com.google.protobuf</groupId> <artifactId>p
HBase Spark错误记录
BigData私房菜
04-24 846
遇到的错误记录 1. Detected both log4j-over-slf4j.jar AND slf4j-log4j12.jar on the class path, preempting StackOverflow 这个错误一直解决不了，pom文件里添加apach的依赖和阿里的云hbase依赖都解决不了，以exclude的形式将其排除也不行，后来找到jdk的路径，之前添加了排除s...
java.lang.BootstrapMethodError: java.lang.IllegalAccessError:
02-24
具体来说，在 Spring Boot 项目里启动过程中抛出 `java.lang.IllegalAccessError: javax/el/Util` 的情形下，可能是 Hibernate Validator 组件与其内部 EL 实现之间的协作出现了障碍。鉴于此，可以通过显式排除过高...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值