java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString

最新推荐文章于 2023-05-11 00:43:08 发布
原创 最新推荐文章于 2023-05-11 00:43:08 发布 · 3w 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#hadoop #spark #hbase #bug

本文详细介绍了Hadoop和Spark在操作HBase时遇到的Java.lang. IllegalAccessError问题的根本原因，并提供了在YARN和Spark环境下解决此问题的具体步骤和配置方法。

hadoop mr 或者 spark 操作 hbase时候就出现这个错误
这是hbase的bug,可在jira上看到该问题:https://issues.apache.org/jira/browse/HBASE-10304

报错信息:

15/08/17 19:28:33 ERROR yarn.ApplicationMaster: User class threw exception: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString ca 
nnot access its superclass com.google.protobuf.LiteralByteString 
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString 
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.translateException(RpcRetryingCaller.java:210) 
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:121) 
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90) 
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:264) 
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:169) 
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:164) 
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:107) 
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:736) 
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:178) 
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:82) 
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isTableAvailable(HConnectionManager.java:962) 
        at org.apache.hadoop.hbase.client.HBaseAdmin.isTableAvailable(HBaseAdmin.java:1081) 
        at org.apache.hadoop.hbase.client.HBaseAdmin.isTableAvailable(HBaseAdmin.java:1089) 
        at com.umeng.dp.yuliang.play.HBaseToES$.main(HBaseToES.scala:28) 
        at com.umeng.dp.yuliang.play.HBaseToES.main(HBaseToES.scala) 
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
        at java.lang.reflect.Method.invoke(Method.java:606) 
        at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:483) 
Caused by: java.lang.IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString 
        at java.lang.ClassLoader.defineClass1(Native Method) 
        at java.lang.ClassLoader.defineClass(ClassLoader.java:800) 
        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) 
        at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) 
        at java.net.URLClassLoader.access$100(URLClassLoader.java:71) 
        at java.net.URLClassLoader$1.run(URLClassLoader.java:361) 
        at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
        at java.security.AccessController.doPrivileged(Native Method) 
        at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
        at java.lang.ClassLoader.loadClass(ClassLoader.java:425) 
        at java.lang.ClassLoader.loadClass(ClassLoader.java:358) 
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildRegionSpecifier(RequestConverter.java:930) 
        at org.apache.hadoop.hbase.protobuf.RequestConverter.buildScanRequest(RequestConverter.java:434) 
        at org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:297) 
        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:157) 
        at org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:57) 
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:114) 
        ... 18 more

hadoop yarn 解决方案:

  1. 提交作业方式

    $ export HADOOP_CLASSPATH="/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar"
$ ./hadoop-2.2.0/bin/hadoop --config /home/stack/conf_hadoop/ jar ./hbase/hbase-assembly/target/hbase-0.99.0-SNAPSHOT-job.jar  org.apache.hadoop.hbase.mapreduce.RowCounter usertable

  2. 增加HADOOP_CLASSPATH到linux环境变量中
    增加如下内容到bashrc 或者 bash_profile 或者 profile ,这样是linux环境变量中就行
    export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar

spark 解决方案:

  1. 提交作业方式
    -conf 增加spark.driver.extraClassPath & spark.executor.extraClassPath

    spark-submit --class com.umeng.dp.yuliang.play.HBaseToES --master yarn-cluster --conf "spark.driver.extraClassPath=/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar" --conf "spark.executor.extraClassPath=/home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar"   --jars /home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar ScalaMR-0.0.1-jar-with-dependencies.jar

  2. 增加如下配置到$SPARK_HOME/conf/spark-defaults.conf文件

    spark.driver.extraClassPath /home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar
spark.executor.extraClassPath /home/cluster/apps/hbase/lib/hbase-protocol-0.98.1-cdh5.1.0.jar

spark maillist:
http://apache-spark-user-list.1001560.n3.nabble.com/java-lang-IllegalAccessError-class-com-google-protobuf-HBaseZeroCopyByteString-cannot-access-its-supg-tc24303.html
尊重原创，未经允许不得转载：
http://blog.csdn.net/stark_summer/article/details/47750017

HBASE: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.<init>
飞流
08-31 3万+
hbase查询时，出现以下异常： Caused by: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.IllegalAccessError: tried to access method com.google.common.base.Stopwatch.()V from class org.apache.hadoop.hbase
java.lang.IllegalAccessError: com/google/protobuf/HBaseZeroCopyByteString
q79969786的专栏
08-30 3422
14/08/30 19:52:37 INFO zookeeper.ClientCnxn: Socket connection established to n3.example.com/192.168.137.103:2181, initiating session 14/08/30 19:52:37 INFO zookeeper.ClientCnxn: Session establishmen
IllegalAccessError HBaseZeroCopyByteString
04-07 174
obuf.LiteralByteString  这个是protobuf版本不一致造成的，但是缺少hbase jar也会报错
java.lang.IllegalAccessError: class javax.activation.SecuritySupport12 cannot access its superclass
runfeel
07-12 754
最近加入新的项目组，eclipse + tomcat7 + spring +ibatis + restful 遇到了这样的问题， 说是不能访问父类，我一开始以为是版本的原因，但是久经更改，错误依然，实在累了， 最终的解决办法是我把SecuritySupport12 所在的jar包activation.jar 删除了，这样在tomcat就可以发布了， 但是这终究不是很好的解决办法，也只能先暂时...
java.long.NoClassDefFoundError:com/google/protobuf/LiteralByteString
mmj123321123的博客
03-25 1500
flink实时数据仓库dwd层从kafka中拿数据，经过FlinkAPP消费，维度数据存放到hbase，事实表存到kafka
"org.springframework.beans.factory.BeanCreationException","error.message":"Error creating bean with name 'withDrawRewardImpl': Injection of resource dependencies failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'cn.migu.wheat.consumer.ConsumerProxy#23': Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [java.lang.Object]: Factory method 'getProxyInstance' threw exception; nested exception is java.lang.IllegalAccessError: class cn.migu.reading.server.readduration.ImportDurationBlackListCptsRequest tried to access method 'com.google.protobuf.LazyStringArrayList com.google.protobuf.LazyStringArrayList.emptyList()' (cn.migu.reading.server.readduration.ImportDurationBlackListCptsRequest and com.google.protobuf.LazyStringArrayList are in unnamed module of loader 'app')", 翻译成中文
06-18
###解决方案步骤1.**升级Protobuf版本**：在`pom.xml`中，将protobuf-java升级到最新稳定版（如3.22.2）：```xml<dependency><groupId>com.google.protobuf</groupId><artifactId>protobuf-java</artifactId>...
Exception in thread "main" java.lang.IllegalAccessError: class org.apache.spark.storage.StorageUtils$ (in unnamed module @0x5be1d0a4) cannot access class sun.nio.ch.DirectBuffer (in module java.base) because module java.base does not export sun.nio.ch to unnamed module @0x5be1d0a4
最新发布
11-12
首先，用户的问题是关于解决Spark程序中的特定错误：'Exception in thread "main" java.lang.IllegalAccessError: class org.apache.spark.storage.StorageUtils$ (in unnamed module @0x5be1d0a4) cannot access ...
6月 25, 2025 9:26:40 下午 com.google.testing.platform.main.MainKt$main$2 uncaughtException 严重: Fatal error while executing main with args: --proto_config=C:\Users\bowen.xiao\.android\utp\runnerConfig12320517331293124685.pb --proto_server_config=C:\Users\bowen.xiao\.android\utp\serverConfig104787935992664716.pb java.lang.IllegalAccessError: class com.google.protobuf.GeneratedMessageV3 tried to access method 'boolean com.google.protobuf.CodedInputStream.shouldDiscardUnknownFields()' (com.google.protobuf.GeneratedMessageV3 is in unnamed module of loader java.net.URLClassLoader @2102a4d5; com.google.protobuf.CodedInputStream is in unnamed module of loader 'app') at com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:325) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry.<init>(AppProcessesProto.java:119) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry.<init>(AppProcessesProto.java:54) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry$1.parsePartialFrom(AppProcessesProto.java:770) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry$1.parsePartialFrom(AppProcessesProto.java:764) at com.google.protobuf.CodedInputStream$ArrayDecoder.readMessage(CodedInputStream.java:980) at com.android.server.adb.protos.AppProcessesProto$AppProcesses.<init>(AppProcessesProto.java:871) at com.android.server.adb.protos.AppProcessesProto$AppProcesses.<init>(AppProcessesProto.java:821) at com.android.server.adb.protos.AppProcessesProto$AppProcesses$1.parsePartialFrom(AppProcessesProto.java:1558) at com.android.server.adb.protos.AppProcessesProto$AppProcesses$1.parsePartialFrom(AppProcessesProto.java:1552) at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:111) at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:126) at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:20) at com.android.server.adb.protos.AppProcessesProto$AppProcesses.parseFrom(AppProcessesProto.java:1014) at com.android.ddmlib.internal.DeviceClientMonitorTask$TrackAppProcessor.processIncomingData(DeviceClientMonitorTask.java:498) at com.android.ddmlib.internal.DeviceClientMonitorTask.run(DeviceClientMonitorTask.java:245) at java.base/java.lang.Thread.run(Thread.java:842) 6月 25, 2025 9:26:40 下午 com.google.testing.platform.RunnerImpl stop 严重: Termination signal received. Stopping test execution. 6月 25, 2025 9:26:40 下午 com.google.testing.platform.runtime.android.driver.AndroidInstrumentationDriver cancel 警告: Cancellation triggered in Android Instrumentation Driver. 6月 25, 2025 9:26:40 下午 com.google.testing.platform.RunnerImpl stop 警告: Waiting for UTP runner to exit... 6月 25, 2025 9:26:40 下午 com.google.testing.platform.RunnerImpl stop 严重: Aborted execution. Interim results: test_status: ABORTED
06-26
我们面对的是一个Android测试执行过程中的IllegalAccessError异常，具体错误信息是："IllegalAccessErrorcom.google.protobuf.GeneratedMessageV3accessmethodCodedInputStreamshouldDiscardUnknownFields"这个错误...
> Task :scripts:connectedDebugAndroidTest 4�� 28, 2025 7:23:36 ���� com.google.testing.platform.main.MainKt$main$2 uncaughtException ����: Fatal error while executing main with args: --proto_config=C:\Users\bowen.xiao\.android\utp\runnerConfig13203883161098104307.pb --proto_server_config=C:\Users\bowen.xiao\.android\utp\serverConfig14046213769878603943.pb java.lang.IllegalAccessError: class com.google.protobuf.GeneratedMessageV3 tried to access method 'boolean com.google.protobuf.CodedInputStream.shouldDiscardUnknownFields()' (com.google.protobuf.GeneratedMessageV3 is in unnamed module of loader java.net.URLClassLoader @77b7ffa4; com.google.protobuf.CodedInputStream is in unnamed module of loader 'app') at com.google.protobuf.GeneratedMessageV3.parseUnknownField(GeneratedMessageV3.java:325) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry.<init>(AppProcessesProto.java:119) at com.android.server.adb.protos.AppProcessesProto$ProcessEntry.<init>(AppProcessesPro
04-29
好的，我现在需要帮助用户解决在执行Task :scripts:connectedDebugAndroidTest时出现的java.lang.IllegalAccessError错误，特别是涉及com.google.protobuf.GeneratedMessageV3和CodedInputStream ...
nested exception is java.lang.IllegalAccessError: tried to access method net.sf.ehcache.CacheManager.()V from class org.hibernat
大树下那片阴凉
06-16 5838
今天配置struts2+hibernate3+spring2时出现这种问题,控制台输出了java.lang.IllegalAccessError: tried to access method net.sf.ehcache.CacheManager.()V from class org.hibernate.cache.EhCacheProvider上网查了一下.原来是JAR文件版本问题 
java报错--＞java.lang.IllegalAccessError
Hitvz的博客
05-11 3891
在gradle中运行main方法突然出现如下错误看报错是资源访问不了，因为用了jdk9之后的jdk版本，会有访问问题，但是我写main方法又不是模块，知道要加jvm参数，但我用的gradle，就很迷茫，怎么设置参数。
JAVA错误解决：tomcat启动错误（java.lang.IllegalAccessError: class javax.xml.parsers.SecuritySupport12 cannot ）
wwpp1的博客
11-21 3579
解决方案：在web.xml中增加信息：metadata-complete="true"  错误如下： java.lang.IllegalAccessError: class javax.xml.parsers.SecuritySupport12 cannot access its superclass javax.xml.parsers.SecuritySupport at java.l...
IllegalAccessError
teasp的专栏
05-17 952
    今天在工作中遇到了这样一个Error，IllegalAccessError。java.lang.IllegalAccessError: class sun.reflect.GeneratedConstructorAccessor3 cannot access its superclass sun.reflect.ConstructorAccessorImpl。本系统使用了OSGI，初步分析...
ZeroCopyLiteralByteString cannot access superclass
宏轩ＪＡＶＡ技术专栏
04-23 2411
问题描述 在HBase上运行MapReduce作业时，报如下异常：IllegalAccessError: class com.google.protobuf.HBaseZeroCopyByteString cannot access its superclass com.google.protobuf.LiteralByteString 使用HBase环境如下：CDH5.0.1, HB
记一次，继承出现的问题“cannot access its superclass”
大鲨鱼丶
06-08 2010
记一次，spring-boot-devtools引发的类加载器问题
记一次解决bug（java.lang.IllegalAccessError: tried to access method ）的过程思考与总结
YvesHe的专栏
08-24 2万+
前言 周六加班，运行idea报错。 严重: Exception sending context initialized event to listener instance of class org.springframework.web.context.ContextLoaderListener java.lang.IllegalAccessError: tried to access method com.i61.common.db.DbManager.getNameDbMap()Ljava.
Caused by: java.lang.ClassNotFoundException常见异常解决方法
热门推荐
关注我看更多东西
05-19 9万+
常见的Caused by: java.lang.ClassNotFoundException异常原因
java.lang.BootstrapMethodError: java.lang.IllegalAccessError:
02-24
### Java 中 `BootstrapMethodError` 和 `IllegalAccessError` 错误分析 #### 关于 `BootstrapMethodError` `BootstrapMethodError` 是一种在运行时发生的错误，通常发生在使用了新的字节码指令（如 invokedynamic），而 JVM 或类库未能正确解析这些新特性的情况。此错误可能由多种因素引起： - **JVM 版本不兼容**：如果应用程序使用了较新的 Java 字节码功能（例如 Lambda 表达式或方法句柄），但在较低版本的 JVM 上执行，则可能会遇到此类异常[^2]。 对于该问题的一个常见场景是在 Spark 应用程序中提交作业时报错——由于 Maven 构建工具配置不当导致项目依赖项中的 Scala 版本与集群环境所使用的 Scala 版本存在差异。这会使得某些特定操作无法正常工作，进而触发 `BootstrapMethodError` 异常。解决办法是调整 POM 文件内的依赖声明来匹配目标平台上的 Scala 发行版，并重新构建应用包后再部署至生产环境中。 ```xml <properties> <scala.version>2.12.10</scala.version> </properties> <dependencies> <!-- Ensure all scala dependencies match the specified version --> ... </dependencies> ``` #### 针对 `IllegalAccessError` 另一方面，`IllegalAccessError` 属于链接阶段产生的严重错误之一，表明尝试访问某个非法成员（字段、构造器或方法）。这类情况往往源于模块化系统的引入以及反射机制的应用不当所致；亦或是不同框架间相互作用引发冲突的结果。具体来说，在 Spring Boot 项目里启动过程中抛出 `java.lang.IllegalAccessError: javax/el/Util` 的情形下，可能是 Hibernate Validator 组件与其内部 EL 实现之间的协作出现了障碍。鉴于此，可以通过显式排除过高版本的验证器组件并替换为更稳定的替代品来进行修复[^4]。 ```xml <!-- Exclude problematic hibernate-validator from spring-boot-starter-web --> <dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-web</artifactId> <exclusions> <exclusion> <groupId>org.hibernate.validator</groupId> <artifactId>hibernate-validator</artifactId> </exclusion> </exclusions> </dependency> <!-- Optionally add a compatible validator dependency --> <dependency> <groupId>javax.validation</groupId> <artifactId>validation-api</artifactId> <version>${validation.api.version}</version> </dependency> ``` 上述措施有助于缓解因第三方库之间潜在的二进制接口变化所带来的影响，从而保障整个 Web 应用能够顺利初始化完成。
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值