java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V

2019独角兽企业重金招聘Python工程师标准>>>

 

首先这个错误，一定是发生了Jar冲突

Jar冲突指的是相同的类，在不同的Jar中出现，这就早晨了 Jar冲突

 

一、错误信息

06-08 14:18:26 [ERROR][job.RunnableJobRunner(67)] java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
        at cc.blueview.data.job.task.kol.FansAnalyeJob.fansBase(FansAnalyeJob.java:403)
        at cc.blueview.data.job.task.kol.FansAnalyeJob.runJob(FansAnalyeJob.java:143)
        at cc.blueview.data.job.task.kol.FansAnalyeJob.call(FansAnalyeJob.java:935)
        at cc.blueview.dmp.schedule.job.RunnableJob.start(RunnableJob.java:31)
        at cc.blueview.dmp.schedule.job.RunnableJobRunner.call(RunnableJobRunner.java:63)
        at cc.blueview.dmp.lego.task.LegoThreadPoolScheduler$2.run(LegoThreadPoolScheduler.java:190)
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)

06-08 14:18:26 [INFO][listener.JobStatusListener(104)] stop job, jobId: kolFansAnalyeJob Stop Status: FAILED
06-08 14:18:26 [ERROR][support.TaskUtils$LoggingErrorHandler(95)] Unexpected error occurred in scheduled task.
java.lang.IllegalStateException: java.lang.IllegalStateException: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
        at cc.blueview.dmp.lego.task.LegoThreadPoolScheduler$2.run(LegoThreadPoolScheduler.java:192)
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
        at cc.blueview.dmp.schedule.job.RunnableJobRunner.call(RunnableJobRunner.java:80)
        at cc.blueview.dmp.lego.task.LegoThreadPoolScheduler$2.run(LegoThreadPoolScheduler.java:190)
        ... 8 more
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V
        at cc.blueview.data.job.task.kol.FansAnalyeJob.fansBase(FansAnalyeJob.java:403)
        at cc.blueview.data.job.task.kol.FansAnalyeJob.runJob(FansAnalyeJob.java:143)
        at cc.blueview.data.job.task.kol.FansAnalyeJob.call(FansAnalyeJob.java:935)
        at cc.blueview.dmp.schedule.job.RunnableJob.start(RunnableJob.java:31)
        at cc.blueview.dmp.schedule.job.RunnableJobRunner.call(RunnableJobRunner.java:63)
        ... 9 more

二、解决方案

沿着线索找下去，发现，spark安装目录的lib下面有个spark-examples-1.4.1-hadoop2.2.0.jar，里面有

org.apache.hadoop.hbase.client.Scan，这个类使用的hbase-client-0.98.7，而在spark的driver端，同样

有hbase-client-1.2.5，里面有org.apache.hadoop.hbase.client.Scan，这样就造成了冲突

转载于:https://my.oschina.net/momisabuilder/blog/917275