![](https://img-blog.csdnimg.cn/20201014180756780.png?x-oss-process=image/resize,m_fixed,h_64,w_64)
hadoop
谷雨hadoop
qq号648380139,欢迎大数据爱好者交流
展开
-
hbase ‘ Failed deleting my ephemeral node’解决
Failed deleting my ephemeral nodeorg.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode for /hbase/rs/0.0.0.0,60020,1409541047246 at org.apache.zookeeper.KeeperException.crea原创 2014-09-01 11:49:53 · 2597 阅读 · 0 评论 -
hdfs dfsadmin -rollingUpgrade解读
hdfs dfsadmin -rollingUpgrade prepare 源码中是CheckpointSignature rollEditLog() throws IOException { getEditLog().rollEditLog(); // Record this log segment ID in all of the storage directori原创 2016-07-14 17:57:17 · 1169 阅读 · 0 评论 -
StandbyException
16/07/15 16:16:48 INFO retry.RetryInvocationHandler: Exception while invoking getFileInfo of class ClientNamenodeProtocolTranslatorPB over hdfs01.beta1.fn/10.202.249.230:9000 after 2 fail over attempt原创 2016-07-15 16:45:44 · 622 阅读 · 0 评论 -
hadoop-2.5.0-cdh5.3.0 HA在线升级
本文升级到hadoop2.61、hadoop升级前准备,namenode元数据和配置备份等信息参考上篇文章:http://blog.csdn.net/linux_ja/article/details/519084662、备份信息:hdfs dfsadmin -rollingUpgrade preparehdfs dfsadmin -rollingUpgrade quer原创 2016-07-15 15:16:38 · 2019 阅读 · 0 评论 -
hive和hbase集成异常
一、偶尔出现两个异常Error: java.lang.IllegalArgumentException: Illegal character code:-1, at 0. User-space table qualifiers can only contain 'alphanumeric characters': i.e. [a-zA-Z_0-9-.]: � at org.apach原创 2016-07-19 16:51:25 · 7188 阅读 · 1 评论 -
datanode Bad connect ack with firstBadLink
1、每次启动job很慢并有异常信息:ERROR - java.io.IOException: Bad connect ack with firstBadLink as 10.21.232.114:5001023-08-2016 14:13:21 CST import_ucord01_order_discount ERROR - at org.apache.hadoop.hdfs.DF原创 2016-08-23 16:58:10 · 1542 阅读 · 0 评论 -
sqoop导数据到hive失败
1、sqoop异常现象:FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. InvalidObjectException(message:There is no database named dw)2、hive.log: 2016-08-23 16:1原创 2016-08-23 16:59:48 · 9857 阅读 · 0 评论 -
mapreduce job任务非常慢
1、application日志2016-08-11 14:48:15,174 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping down all scheduled reduces:02016-08-11 14:48:15,174 INFO [原创 2016-08-11 22:23:08 · 7934 阅读 · 0 评论 -
flume sink hdfs异常
1、异常消息如下:016-08-26 14:19:17,704 (hdfs-sink1-call-runner-2) [ERROR - org.apache.flume.sink.hdfs.AbstractHDFSWriter.hflushOrSync(AbstractHDFSWriter.java:267)] Error while trying to hflushOrSync!2016-原创 2016-08-26 15:47:16 · 9175 阅读 · 1 评论 -
Hadoop 2.3.0-cdh5.0.2升级Hadoop 2.5.0-cdh5.3.1
一、准备升级包for line in `cat /home/hadoop/platform/hadoop.list|awk '{print $1}'`; do echo $line;ssh $line "/bin/mkdir /home/hadoop/platform";donefor line in `cat /home/hadoop/platform/hadoop.list|a原创 2016-07-14 13:41:08 · 1232 阅读 · 0 评论 -
hive on tez遇到问题
一、环境hive1.3.1,tez0.5.0二、运行异常如下:return code -101 from org.apache.hadoop.hive.ql.exec.tez.TezTask. org.apache.tez.mapreduce.hadoop.MRHelpers.getBaseMRConfiguration(Lorg/apache/hadoop/conf/Configurat原创 2015-07-14 14:59:39 · 3184 阅读 · 0 评论 -
hbase实现分页注意事项
使用PageFilter的时候,因为返回的是每个region的前原创 2014-09-01 16:56:38 · 679 阅读 · 0 评论 -
用java代码实现sqoop代码
1,客户端编写:原创 2014-09-01 16:47:32 · 1276 阅读 · 0 评论 -
hbase0.98 endpoint实现group分组求和代码
1,制作test.proto文件:option java_package = "com.coprocessor.group.generated"; option java_outer_classname = "GroupProtos"; option java_generic_services = true; option java_generate_equals_and_h原创 2014-09-01 16:24:34 · 2531 阅读 · 1 评论 -
4, org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, in
1, org.apache.hadoop.security.AccessControlException:Permission denied: user=root, access=WRITE,inode="/user":hdfs:supergroup:drwxr-xr-x 错误解决:a,修改hdfs-site.xml: dfs.permissions原创 2014-09-16 14:08:44 · 1713 阅读 · 0 评论 -
java.io.IOException: No FileSystem for scheme: hdfs异常解决
Configuration conf = new Configuration();conf.set("fs.hdfs.impl",org.apache.hadoop.hdfs.DistributedFileSystem.class.getName());fs = FileSystem.get(new URI(HDFS_PATH), conf);若出现java.io.IOExc原创 2014-12-15 17:31:51 · 13850 阅读 · 4 评论 -
hbase0.98分布式部署
两个master,两个regionserverhbaes-env.sh:# The directory where pid files are stored. /tmp by default.export HBASE_PID_DIR=/home/master/platform/hbase-0.98.1-cdh5.1.0/pids# Seconds to sleep原创 2015-02-04 19:33:23 · 717 阅读 · 0 评论 -
hive on tez集成
一、1.下载apache-tez-0.5.4-src.tar.gz2.下载apache-maven-3.2.5-bin.tar3.下载protobuf-2.5.0.tar.gz二、安装1.安装mvn,省略2.安装protobuf: a.tar -zxf protobuf-2.5.0.tar.gz命令解压后得到是protobuf-2.5.0的源码, b.原创 2015-07-14 11:14:21 · 1335 阅读 · 0 评论 -
hive1.2以前版本的where条件NullPointerException
1、异常背景:hive版本1.1.0,表是orc格式,使用条件where name in ('支付金额','订单量','客单价','毛利率','全链路达成率','猫超重点商品在架率','基准价毛利率','商品缺货率')2、日志如下:Diagnostic Messages for this Task:Error: java.lang.RuntimeException: org.apache.had...原创 2018-03-20 17:54:26 · 550 阅读 · 0 评论