![](https://img-blog.csdnimg.cn/2019092715111047.png?x-oss-process=image/resize,m_fixed,h_224,w_224)
报错日记
大数据开发过程中遇到的问题汇总
ETL_JJKING
在脱发的路上越走越远~....走越远~.....越远~........远~
展开
-
大数据 排错日记008——Exception in thread main java.lang.NoSuchMethodError: org.apache.spark.internal.config.
报错信息(主要部分)Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.internal.config.package$.SHUFFLE_SPILL_NUM_ELEMENTS_FORCE_SPILL_THRESHOLD()Lorg/apache/spark/internal/config/ConfigEntry; at org.apache.spark.sql.internal.SQLConf$.<ini原创 2020-05-28 10:29:55 · 3064 阅读 · 2 评论 -
大数据 排错日记007——Call From hadoop01/192.168.100.240 to hadoop01:9000 failed on connection exception
报错信息(主要部分)[hadoop@hadoop01 dfs]$ hdfs dfs -mkdir /inputmkdir: Call From hadoop01/192.168.100.240 to hadoop01:9000 failed on connection exception: java.net.ConnectException: 拒绝连接; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused.原创 2020-05-27 18:02:51 · 2897 阅读 · 0 评论 -
大数据 排错日记0006——Connection timed out: no further information
报错信息(主要部分)2020-03-31 15:38:26,551 INFO [org.apache.zookeeper.ClientCnxn] - Opening socket connection to server hadoop11/192.168.100.211:2181. Will not attempt to authenticate using SASL (unknown erro...原创 2020-03-31 16:15:54 · 1562 阅读 · 1 评论 -
大数据 排错日记0005——Provider org.apache.hadoop.hdfs.web.WebHdfsFileSystem could not be instantiated
报错信息(主要部分)java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.hdfs.web.WebHdfsFileSystem could not be instantiated at java.util.ServiceLoader.fail(Servic...原创 2020-03-30 17:12:55 · 3938 阅读 · 1 评论 -
大数据 排错日记0004——Unable to check if JNs are ready for formatting
报错信息(主要部分)20/03/12 15:35:15 INFO ipc.Client: Retrying connect to server: hadoop10/192.168.100.210:8485. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sle...原创 2020-03-12 16:35:47 · 797 阅读 · 0 评论 -
大数据 排错日记0003——org.apache.hadoop.fs.ChecksumException: Checksum error
报错信息 2020-03-05 12:05:57,079 INFO [org.apache.hadoop.conf.Configuration.deprecation] - session.id is deprecated. Instead, use dfs.metrics.session-id2020-03-05 12:05:57,080 INFO [org.apache.hadoop....原创 2020-03-05 12:27:20 · 446 阅读 · 0 评论 -
大数据 排错日记0002—— Problem binding to [hadoop01:8031] java.net.BindException: 无法指定被请求的地址; For more detai
报错信息 2020-02-27 09:27:40,711 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: Initialized queue: default: capacity=1.0, absoluteCapacity=1.0, usedResources=...原创 2020-02-28 09:56:09 · 1918 阅读 · 0 评论 -
大数据 排错日记0001——MapReduce程序
数据类型比较问题先看图第一次执行结果:第二次执行结果:分析: 经两次代码对比不难发现,问题就在于TreeSet的泛型,两次的代码中只有TreeSet的泛型是不一样的,于是乎两次的执行结果中却大不相同。原因查找:  ...原创 2020-02-26 16:32:39 · 105 阅读 · 0 评论