HBase 通过MapReducer 统计数据写入MySQL中报异常:Exception in thread “main“ java.lang.IllegalAccessError

Exception in thread "main" java.lang.IllegalAccessError: class org.apache.hadoop.hdfs.web.HftpFileSystem cannot access its superinterface org.apache.hadoop.hdfs.web.TokenAspect$TokenManagementDelegator

主要异常:org.apache.hadoop.hdfs.web.HftpFileSystem cannot access its superinterface 

异常如下:

Exception in thread "main" java.lang.IllegalAccessError: class org.apache.hadoop.hdfs.web.HftpFileSystem cannot access its superinterface org.apache.hadoop.hdfs.web.TokenAspect$TokenManagementDelegator
	at java.lang.ClassLoader.defineClass1(Native Method)
	at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
	at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
	at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
	at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
	at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:370)
	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
	at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
	at org.apache.hadoop.fs.FileSystem.loadFileSystems(FileSystem.java:3217)
	at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:3262)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3301)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3352)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3320)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479)
	at org.apache.hadoop.fs.FileSystem.getLocal(FileSystem.java:435)
	at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJarsForClasses(TableMapReduceUtil.java:924)
	at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addHBaseDependencyJars(TableMapReduceUtil.java:815)
	at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:871)
	at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableMapperJob(TableMapReduceUtil.java:214)
	at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableMapperJob(TableMapReduceUtil.java:170)
	at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableMapperJob(TableMapReduceUtil.java:302)
	at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initTableMapperJob(TableMapReduceUtil.java:94)
	at com.atguigu.ct.analysis.tool.AnalysisTextTool.run(AnalysisTextTool.java:29)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
	at com.atguigu.ct.analysis.AnalysisData.main(AnalysisData.java:12)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:318)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:232)

异常原因:

因为hadoop-hdfs-x.x.x.jar发送冲突导致的  详细请参考文章:参考原文章 (https://github.com/Intel-bigdata/HiBench/issues/466)

或者参考引用文章：https://blog.csdn.net/qq_18945757/article/details/105918156

异常解决:

解决办法是，在IDEA中导出jar包之前，File->Project Structure->Artifacts->xxx.jar->Output Layout，将hadoop-hdfs-x.x.x.jar这个文件去除，就行了。

<dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <version>2.3.5</version>
            <exclusions>
                <exclusion>
                    <groupId>org.apache.hadoop</groupId>
                    <artifactId>hadoop-hdfs</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase-server -->
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-server</artifactId>
            <version>2.3.5</version>
            <!--
             解决打包错误：Failure to find org.glassfish:javax.el:pom:3.0.1-b08-SNAPSHOT
             -->
            <exclusions>
                <exclusion>
                    <groupId>org.glassfish</groupId>
                    <artifactId>javax.el</artifactId>
                </exclusion>
                    <!--
            打包前去除hdfs相关jar包
             -->
                <exclusion>
                    <groupId>org.apache.hadoop</groupId>
                    <artifactId>hadoop-hdfs</artifactId>
                </exclusion>
            </exclusions>
        </dependency>


<dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-mapreduce</artifactId>
            <version>2.3.5</version>
            <exclusions>
                <exclusion>
                    <groupId>org.apache.hadoop</groupId>
                    <artifactId>hadoop-hdfs</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

执行成功！！！

mode : false
2021-05-09 13:49:39,229 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1647)) -  map 0% reduce 0%
2021-05-09 13:49:52,036 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1647)) -  map 33% reduce 0%
2021-05-09 13:49:56,118 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1647)) -  map 67% reduce 0%
2021-05-09 13:49:58,148 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1647)) -  map 100% reduce 0%
2021-05-09 13:50:01,192 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1647)) -  map 100% reduce 100%
2021-05-09 13:50:02,211 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1658)) - Job job_1620531497020_0001 completed successfully
2021-05-09 13:50:02,317 INFO  [main] mapreduce.Job (Job.java:monitorAndPrintJob(1665)) - Counters: 65
	File System Counters
		FILE: Number of bytes read=24186
		FILE: Number of bytes written=1855271
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=638
		HDFS: Number of bytes written=0
		HDFS: Number of read operations=6
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=0
	Job Counters 
		Killed map tasks=1
		Launched map tasks=6
		Launched reduce tasks=1
		Data-local map tasks=6
		Total time spent by all maps in occupied slots (ms)=79781
		Total time spent by all reduces in occupied slots (ms)=6475
		Total time spent by all map tasks (ms)=79781
		Total time spent by all reduce tasks (ms)=6475
		Total vcore-milliseconds taken by all map tasks=79781
		Total vcore-milliseconds taken by all reduce tasks=6475
		Total megabyte-milliseconds taken by all map tasks=81695744
		Total megabyte-milliseconds taken by all reduce tasks=6630400
	Map-Reduce Framework
		Map input records=155
		Map output records=930
		Map output bytes=22320
		Map output materialized bytes=24216
		Input split bytes=638
		Combine input records=0
		Combine output records=0
		Reduce input groups=521
		Reduce shuffle bytes=24216
		Reduce input records=930
		Reduce output records=521
		Spilled Records=1860
		Shuffled Maps =6
		Failed Shuffles=0
		Merged Map outputs=6
		GC time elapsed (ms)=5782
		CPU time spent (ms)=21900
		Physical memory (bytes) snapshot=2693857280
		Virtual memory (bytes) snapshot=18332504064
		Total committed heap usage (bytes)=2427977728
		Peak Map Physical memory (bytes)=499822592
		Peak Map Virtual memory (bytes)=2653581312
		Peak Reduce Physical memory (bytes)=173019136
		Peak Reduce Virtual memory (bytes)=2574319616
	HBaseCounters
		BYTES_IN_REMOTE_RESULTS=0
		BYTES_IN_RESULTS=70525
		MILLIS_BETWEEN_NEXTS=17960
		NOT_SERVING_REGION_EXCEPTION=0
		REGIONS_SCANNED=6
		REMOTE_RPC_CALLS=0
		REMOTE_RPC_RETRIES=0
		ROWS_FILTERED=0
		ROWS_SCANNED=155
		RPC_CALLS=6
		RPC_RETRIES=0
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	File Input Format Counters 
		Bytes Read=0
	File Output Format Counters 
		Bytes Written=0
2执行完成!!!