hadoop之MapReduce调用R的一次失败的总结~(续五)

最新推荐文章于 2020-11-17 11:18:13 发布
最新推荐文章于 2020-11-17 11:18:13 发布
阅读量1.2k 收藏
点赞数
分类专栏： hadoop
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/yibei8811/article/details/47105407
版权

前面说过R端抛出异常

rhbase<hbScannerOpenFilterEx>:: (IOError) Default TException


程序第一次执行MapReduce（6分5秒）的时候是不抛出异常的，本以为第二次执行MapReduce就会抛出，测试情况并不是如此。

实测中，如果第2次MapReduce等第一次结束后，立即执行，一段时间后MapReduce才会抛出异常，接着结束。耗时5分5秒，因为异常了所以很快的结束了。

换句话说，是在11分10秒的时候抛出异常，因为一旦发生异常，程序结束的会很快。

考虑到程序是通过thrift抛出连接Hbase，检查thrift日志，信息如下： 


[hadoop@bd110 logs]$ less -M hbase-hadoop-thrift-bd110.log 
2015年 07月 28日 星期二 15:01:01 CST Terminating thrift
2015年 07月 28日 星期二 15:01:09 CST Starting thrift on bd110
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 125504
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 32768
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 32768
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
2015-07-28 15:01:09,834 INFO  [main] util.VersionInfo: HBase 1.1.1
2015-07-28 15:01:09,834 INFO  [main] util.VersionInfo: Source code repository git://hw11397.local/Volumes/hbase-1.1.1RC0/hbase revision=d0a115a7267f54e01c72c603ec53e91ec418292f
2015-07-28 15:01:09,834 INFO  [main] util.VersionInfo: Compiled by ndimiduk on Tue Jun 23 14:44:07 PDT 2015
2015-07-28 15:01:09,834 INFO  [main] util.VersionInfo: From source with checksum 6e2d8cecbd28738ad86daacb25dc467e
2015-07-28 15:01:10,070 INFO  [main] thrift.ThriftServerRunner: Using default thrift server type
2015-07-28 15:01:10,070 INFO  [main] thrift.ThriftServerRunner: Using thrift server type threadpool
2015-07-28 15:01:10,305 INFO  [main] impl.MetricsConfig: loaded properties from hadoop-metrics2-hbase.properties
2015-07-28 15:01:10,345 INFO  [main] impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s).
2015-07-28 15:01:10,345 INFO  [main] impl.MetricsSystemImpl: HBase metrics system started
2015-07-28 15:01:10,419 INFO  [main] mortbay.log: Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2015-07-28 15:01:10,422 WARN  [main] http.HttpRequestLog: Jetty request log can only be enabled using Log4j
2015-07-28 15:01:10,427 INFO  [main] http.HttpServer: Added global filter 'safety' (class=org.apache.hadoop.hbase.http.HttpServer$QuotingInputFilter)
2015-07-28 15:01:10,429 INFO  [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context thrift
2015-07-28 15:01:10,429 INFO  [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context static
2015-07-28 15:01:10,429 INFO  [main] http.HttpServer: Added filter static_user_filter (class=org.apache.hadoop.hbase.http.lib.StaticUserWebFilter$StaticUserFilter) to context logs
2015-07-28 15:01:10,439 INFO  [main] http.HttpServer: Jetty bound to port 9095
2015-07-28 15:01:10,439 INFO  [main] mortbay.log: jetty-6.1.26
2015-07-28 15:01:10,460 WARN  [main] mortbay.log: Can't reuse /tmp/Jetty_0_0_0_0_9095_thrift____.vqpz9l, using /tmp/Jetty_0_0_0_0_9095_thrift____.vqpz9l_6567097626148371637
2015-07-28 15:01:10,649 INFO  [main] mortbay.log: Started SelectChannelConnector@0.0.0.0:9095
2015-07-28 15:01:10,664 INFO  [main] thrift.ThriftServerRunner: starting TBoundedThreadPoolServer on /0.0.0.0:9090; min worker threads=16, max worker threads=1000, max queued requests=1000
2015-07-28 15:02:17,467 INFO  [thrift-worker-0] zookeeper.RecoverableZooKeeper: Process identifier=hconnection-0x69eeaa65 connecting to ZooKeeper ensemble=bd107:2181,bd108:2181,bd110:2181
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.6-1569965, built on 02/20/2014 09:09 GMT
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:host.name=bd110
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:java.version=1.7.0_67
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:java.home=/usr/local/jdk1.7/jre
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:java.class.path=/home/hadoop/hbase-1.1.1/conf:/usr/local/jdk1.7/lib/tools.jar:/home/hadoop/hbase-1.1.1:/home/hadoop/hbase-1.1.1/lib/activation-1.1.jar:/home/hadoop/hbase-1.1.1/lib/aopalliance-1.0.jar:/home/hadoop/hbase-1.1.1/lib/apacheds-i18n-2.0.0-M15.jar:/home/hadoop/hbase-1.1.1/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hadoop/hbase-1.1.1/lib/api-asn1-api-1.0.0-M20.jar:/home/hadoop/hbase-1.1.1/lib/api-util-1.0.0-M20.jar:/home/hadoop/hbase-1.1.1/lib/asm-3.1.jar:/home/hadoop/hbase-1.1.1/lib/avro-1.7.4.jar:/home/hadoop/hbase-1.1.1/lib/commons-beanutils-1.7.0.jar:/home/hadoop/hbase-1.1.1/lib/commons-beanutils-core-1.8.0.jar:/home/hadoop/hbase-1.1.1/lib/commons-cli-1.2.jar:/home/hadoop/hbase-1.1.1/lib/commons-codec-1.9.jar:/home/hadoop/hbase-1.1.1/lib/commons-collections-3.2.1.jar:/home/hadoop/hbase-1.1.1/lib/commons-compress-1.4.1.jar:/home/hadoop/hbase-1.1.1/lib/commons-configuration-1.6.jar:/home/hadoop/hbase-1.1.1/lib/commons-daemon-1.0.13.jar:/home/hadoop/hbase-1.1.1/lib/commons-digester-1.8.jar:/home/hadoop/hbase-1.1.1/lib/commons-el-1.0.jar:/home/hadoop/hbase-1.1.1/lib/commons-httpclient-3.1.jar:/home/hadoop/hbase-1.1.1/lib/commons-io-2.4.jar:/home/hadoop/hbase-1.1.1/lib/commons-lang-2.6.jar:/home/hadoop/hbase-1.1.1/lib/commons-logging-1.2.jar:/home/hadoop/hbase-1.1.1/lib/commons-math-2.2.jar:/home/hadoop/hbase-1.1.1/lib/commons-math3-3.1.1.jar:/home/hadoop/hbase-1.1.1/lib/commons-net-3.1.jar:/home/hadoop/hbase-1.1.1/lib/disruptor-3.3.0.jar:/home/hadoop/hbase-1.1.1/lib/findbugs-annotations-1.3.9-1.jar:/home/hadoop/hbase-1.1.1/lib/guava-12.0.1.jar:/home/hadoop/hbase-1.1.1/lib/guice-3.0.jar:/home/hadoop/hbase-1.1.1/lib/guice-servlet-3.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-annotations-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-auth-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-client-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-common-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-hdfs-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-mapreduce-client-app-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-mapreduce-client-common-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-mapreduce-client-core-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-mapreduce-client-jobclient-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-mapreduce-client-shuffle-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-yarn-api-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-yarn-client-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-yarn-common-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hadoop-yarn-server-common-2.6.0.jar:/home/hadoop/hbase-1.1.1/lib/hbase-annotations-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-annotations-1.1.1-tests.jar:/home/hadoop/hbase-1.1.1/lib/hbase-client-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-common-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-common-1.1.1-tests.jar:/home/hadoop/hbase-1.1.1/lib/hbase-examples-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-hadoop2-compat-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-hadoop-compat-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-it-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-it-1.1.1-tests.jar:/home/hadoop/hbase-1.1.1/lib/hbase-prefix-tree-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-procedure-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-protocol-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-rest-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-server-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-server-1.1.1-tests.jar:/home/hadoop/hbase-1.1.1/lib/hbase-shell-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/hbase-thrift-1.1.1.jar:/home/hadoop/hbase-1.1.1/lib/htrace-core-3.0.4.jar:/home/hadoop/hbase-1.1.1/lib/htrace-core-3.1.0-incubating.jar:/home/hadoop/hbase-1.1.1/lib/httpclient-4.2.5.jar:/home/hadoop/hbase-1.1.1/lib/httpcore-4.1.3.jar:/home/hadoop/hbase-1.1.1/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hbase-1.1.1/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/hbase-1.1.1/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hbase-1.1.1/lib/jackson-xc-1.9.13.jar:/home/hadoop/hbase-1.1.1/lib/jamon-runtime-2.3.1.jar:/home/hadoop/hbase-1.1.1/lib/jasper-compiler-5.5.23.jar:/home/hadoop/hbase-1.1.1/lib/jasper-runtime-5.5.23.jar:/home/hadoop/hbase-1.1.1/lib/javax.inject-1.jar:/home/hadoop/hbase-1.1.1/lib/java-xmlbuilder-0.4.jar:/home/hadoop/hbase-1.1.1/lib/jaxb-api-2.2.2.jar:/home/hadoop/hbase-1.1.1/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/hbase-1.1.1/lib/jcodings-1.0.8.jar:/home/hadoop/hbase-1.1.1/lib/jersey-client-1.9.jar:/home/hadoop/hbase-1.1.1/lib/jersey-core-1.9.jar:/home/hadoop/hbase-1.1.1/lib/jersey-guice-1.9.jar:/home/hadoop/hbase-1.1.1/lib/jersey-json-1.9.jar:/home/hadoop/hbase-1.1.1/lib/jersey-server-1.9.jar:/home/hadoop/hbase-1.1.1/lib/jets3t-0.9.0.jar:/home/hadoop/hbase-1.1.1/lib/jettison-1.3.3.jar:/home/hadoop/hbase-1.1.1/lib/jetty-6.1.26.jar:/home/hadoop/hbase-1.1.1/lib/jetty-sslengine-6.1.26.jar:/home/hadoop/hbase-1.1.1/lib/jetty-util-6.1.26.jar:/home/hadoop/hbase-1.1.1/lib/joni-2.1.2.jar:/home/hadoop/hbase-1.1.1/lib/jruby-complete-1.6.8.jar:/home/hadoop/hbase-1.1.1/lib/jsch-0.1.42.jar:/home/hadoop/hbase-1.1.1/lib/jsp-2.1-6.1.14.jar:/home/hadoop/hbase-1.1.1/lib/jsp-api-2.1-6.1.14.jar:/home/hadoop/hbase-1.1.1/lib/jsr305-1.3.9.jar:/home/hadoop/hbase-1.1.1/lib/junit-4.11.jar:/home/hadoop/hbase-1.1.1/lib/leveldbjni-all-1.8.jar:/home/hadoop/hbase-1.1.1/lib/libthrift-0.9.0.jar:/home/hadoop/hbase-1.1.1/lib/log4j-1.2.17.jar:/home/hadoop/hbase-1.1.1/lib/metrics-core-2.2.0.jar:/home/hadoop/hbase-1.1.1/lib/netty-3.2.4.Final.jar:/home/hadoop/hbase-1.1.1/lib/netty-all-4.0.23.Final.jar:/home/hadoop/hbase-1.1.1/lib/paranamer-2.3.jar:/home/hadoop/hbase-1.1.1/lib/protobuf-java-2.5.0.jar:/home/hadoop/hbase-1.1.1/lib/servlet-api-2.5-6.1.14.jar:/home/hadoop/hbase-1.1.1/lib/servlet-api-2.5.jar:/home/hadoop/hbase-1.1.1/lib/slf4j-api-1.7.7.jar:/home/hadoop/hbase-1.1.1/lib/slf4j-log4j12-1.7.5.jar:/home/hadoop/hbase-1.1.1/lib/snappy-java-1.0.4.1.jar:/home/hadoop/hbase-1.1.1/lib/spymemcached-2.11.6.jar:/home/hadoop/hbase-1.1.1/lib/xmlenc-0.52.jar:/home/hadoop/hbase-1.1.1/lib/xz-1.0.jar:/home/hadoop/hbase-1.1.1/lib/zookeeper-3.4.6.jar:/home/hadoop/hadoop-2.6.0/etc/hadoop:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-collections-3.2.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jersey-json-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hadoop-auth-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hadoop-annotations-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-codec-1.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/httpcore-4.2.5.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-digester-1.8.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/asm-3.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jettison-1.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jetty-util-6.1.26.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/curator-recipes-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/gson-2.2.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/curator-client-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-lang-2.6.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/curator-framework-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/guava-11.0.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/junit-4.11.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/httpclient-4.2.5.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jetty-6.1.26.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jasper-runtime-5.5.23.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/stax-api-1.0-2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-3.0.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hamcrest-core-1.3.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-net-3.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/servlet-api-2.5.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/xmlenc-0.52.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jsr305-1.3.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-math3-3.1.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/paranamer-2.3.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-el-1.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jasper-compiler-5.5.23.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jsch-0.1.42.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/xz-1.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jsp-api-2.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/zookeeper-3.4.6.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/avro-1.7.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jets3t-0.9.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/activation-1.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-annotations-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-checkstyle-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-client-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-common-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-common-0.98.10-hadoop2-tests.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-examples-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-hadoop2-compat-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-hadoop-compat-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-it-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-it-0.98.10-hadoop2-tests.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-prefix-tree-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-protocol-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-rest-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-server-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-server-0.98.10-hadoop2-tests.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-shell-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-testing-util-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/hbase-thrift-0.98.10-hadoop2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/htrace-core-2.04.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/high-scale-lib-1.1.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/aspectjweaver-1.7.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-dbcp-1.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-fileupload-1.2.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-lang3-3.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/commons-pool-1.5.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/druid-1.0.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jcl-over-slf4j-1.6.6.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/jul-to-slf4j-1.6.6.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/log4jdbc-remix-0.2.7.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/mybatis-3.2.7.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/mybatis-spring-1.2.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/postgresql-9.3-1100-jdbc4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-aop-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-aspects-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-beans-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-context-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-context-support-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-core-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/JRIEngine.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-expression-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-jdbc-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-orm-3.2.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-security-acl-3.2.4.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-security-core-3.2.4.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-security-taglibs-3.2.4.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-security-web-3.2.4.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-tx-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-web-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/spring-webmvc-4.0.6.RELEASE.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/DEEPLAN-MRCOMN-0.0.1-SNAPSHOT.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/JRI.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/lib/REngine.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0-tests.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/hadoop-common-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/common/hadoop-nfs-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/asm-3.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jasper-runtime-5.5.23.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/htrace-core-3.0.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jsr305-1.3.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-el-1.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/jsp-api-2.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-nfs-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0-tests.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/hdfs/hadoop-hdfs-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-collections-3.2.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-json-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-codec-1.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jline-0.9.94.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/asm-3.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jettison-1.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/guava-11.0.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/javax.inject-1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jetty-6.1.26.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-httpclient-3.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/guice-3.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jersey-client-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/servlet-api-2.5.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jsr305-1.3.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/xz-1.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/lib/activation-1.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-api-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-client-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-common-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-registry-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-server-tests-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-common-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/hadoop-annotations-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/junit-4.11.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.6.0-tests.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar:/home/hadoop/hadoop-2.6.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.6.0.jar:/home/hadoop/hadoop-2.6.0/contrib/capacity-scheduler/*.jar
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:java.library.path=/home/hadoop/hadoop-2.6.0/lib/native
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:os.name=Linux
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:os.arch=amd64
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:os.version=3.10.0-123.el7.x86_64
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:user.name=hadoop
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:user.home=/home/hadoop
2015-07-28 15:02:17,472 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Client environment:user.dir=/home/hadoop/hbase-1.1.1/logs
2015-07-28 15:02:17,473 INFO  [thrift-worker-0] zookeeper.ZooKeeper: Initiating client connection, connectString=bd107:2181,bd108:2181,bd110:2181 sessionTimeout=90000 watcher=hconnection-0x69eeaa650x0, quorum=bd107:2181,bd108:2181,bd110:2181, baseZNode=/hbase
2015-07-28 15:02:17,484 INFO  [thrift-worker-0-SendThread(bd110:2181)] zookeeper.ClientCnxn: Opening socket connection to server bd110/192.168.101.110:2181. Will not attempt to authenticate using SASL (unknown error)
2015-07-28 15:02:17,485 INFO  [thrift-worker-0-SendThread(bd110:2181)] zookeeper.ClientCnxn: Socket connection established to bd110/192.168.101.110:2181, initiating session
2015-07-28 15:02:17,513 INFO  [thrift-worker-0-SendThread(bd110:2181)] zookeeper.ClientCnxn: Session establishment complete on server bd110/192.168.101.110:2181, sessionid = 0x24eced771600008, negotiated timeout = 90000
2015-07-28 15:13:42,767 ERROR [thrift-worker-0] thrift.ProcessFunction: Internal error processing scannerOpenWithScan
java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@13656ae rejected from java.util.concurrent.ThreadPoolExecutor@1674ba21[Shutting down, pool size = 7, active threads = 7, queued tasks = 0, completed tasks = 1889]
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:208)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@13656ae rejected from java.util.concurrent.ThreadPoolExecutor@1674ba21[Shutting down, pool size = 7, active threads = 7, queued tasks = 0, completed tasks = 1889]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
        at org.apache.hadoop.hbase.client.ResultBoundedCompletionService.submit(ResultBoundedCompletionService.java:142)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.addCallsForCurrentReplica(ScannerCallableWithReplicas.java:290)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:169)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        ... 20 more
2015-07-28 15:13:43,508 ERROR [thrift-worker-15] thrift.ProcessFunction: Internal error processing scannerOpenWithScan
java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@5e28da27 rejected from java.util.concurrent.ThreadPoolExecutor@1674ba21[Shutting down, pool size = 6, active threads = 6, queued tasks = 0, completed tasks = 1890]
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:208)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@5e28da27 rejected from java.util.concurrent.ThreadPoolExecutor@1674ba21[Shutting down, pool size = 6, active threads = 6, queued tasks = 0, completed tasks = 1890]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
        at org.apache.hadoop.hbase.client.ResultBoundedCompletionService.submit(ResultBoundedCompletionService.java:142)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.addCallsForCurrentReplica(ScannerCallableWithReplicas.java:290)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:169)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        ... 20 more
2015-07-28 15:13:43,693 ERROR [thrift-worker-8] thrift.ProcessFunction: Internal error processing scannerOpenWithScan
java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@139bc376 rejected from java.util.concurrent.ThreadPoolExecutor@1674ba21[Shutting down, pool size = 3, active threads = 3, queued tasks = 0, completed tasks = 1893]
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:208)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@139bc376 rejected from java.util.concurrent.ThreadPoolExecutor@1674ba21[Shutting down, pool size = 3, active threads = 3, queued tasks = 0, completed tasks = 1893]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
        at org.apache.hadoop.hbase.client.ResultBoundedCompletionService.submit(ResultBoundedCompletionService.java:142)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.addCallsForCurrentReplica(ScannerCallableWithReplicas.java:290)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:169)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        ... 20 more
2015-07-28 15:13:43,715 ERROR [thrift-worker-5] thrift.ProcessFunction: Internal error processing scannerOpenWithScan
java.lang.RuntimeException: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@27dee71a rejected from java.util.concurrent.ThreadPoolExecutor@1674ba21[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 1894]
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:208)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.RejectedExecutionException: Task org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture@27dee71a rejected from java.util.concurrent.ThreadPoolExecutor@1674ba21[Shutting down, pool size = 2, active threads = 2, queued tasks = 0, completed tasks = 1894]
        at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
        at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
        at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1372)
        at org.apache.hadoop.hbase.client.ResultBoundedCompletionService.submit(ResultBoundedCompletionService.java:142)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.addCallsForCurrentReplica(ScannerCallableWithReplicas.java:290)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:169)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        ... 20 more
2015-07-28 15:13:43,740 INFO  [ConnectionCache_ChoreService_1] client.ConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x24eced771600008
2015-07-28 15:13:43,765 INFO  [ConnectionCache_ChoreService_1] zookeeper.ZooKeeper: Session: 0x24eced771600008 closed
2015-07-28 15:13:43,765 INFO  [thrift-worker-0-EventThread] zookeeper.ClientCnxn: EventThread shut down
2015-07-28 15:13:43,782 WARN  [thrift-worker-1] thrift.ThriftServerRunner$HBaseHandler: Can't get the location
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:309)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x69eeaa65 closed
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1146)
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:300)
        ... 23 more
2015-07-28 15:13:43,991 WARN  [thrift-worker-4] thrift.ThriftServerRunner$HBaseHandler: Can't get the location
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:309)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x69eeaa65 closed
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1146)
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:300)
        ... 23 more
2015-07-28 15:13:44,023 WARN  [thrift-worker-10] thrift.ThriftServerRunner$HBaseHandler: Can't get the location
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:309)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x69eeaa65 closed
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1146)
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:300)
        ... 23 more
2015-07-28 15:13:44,273 WARN  [thrift-worker-8] thrift.ThriftServerRunner$HBaseHandler: Can't get the location
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:309)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x69eeaa65 closed
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1146)
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:300)
        ... 23 more
2015-07-28 15:13:44,276 WARN  [thrift-worker-8] thrift.ThriftServerRunner$HBaseHandler: Can't get the location
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:309)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x69eeaa65 closed
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1146)
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:300)
        ... 23 more
2015-07-28 15:13:44,779 WARN  [thrift-worker-1] thrift.ThriftServerRunner$HBaseHandler: Can't get the location
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:309)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x69eeaa65 closed
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1146)
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:300)
        ... 23 more
2015-07-28 15:13:44,780 WARN  [thrift-worker-1] thrift.ThriftServerRunner$HBaseHandler: Can't get the location
org.apache.hadoop.hbase.client.RetriesExhaustedException: Can't get the location
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:309)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:153)
        at org.apache.hadoop.hbase.client.ScannerCallableWithReplicas.call(ScannerCallableWithReplicas.java:61)
        at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
        at org.apache.hadoop.hbase.client.ClientScanner.call(ClientScanner.java:320)
        at org.apache.hadoop.hbase.client.ClientScanner.nextScanner(ClientScanner.java:295)
        at org.apache.hadoop.hbase.client.ClientScanner.initializeScannerInConstruction(ClientScanner.java:160)
        at org.apache.hadoop.hbase.client.ClientScanner.<init>(ClientScanner.java:155)
        at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:811)
        at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.scannerOpenWithScan(ThriftServerRunner.java:1451)
        at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
        at com.sun.proxy.$Proxy11.scannerOpenWithScan(Unknown Source)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4609)
        at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$scannerOpenWithScan.getResult(Hbase.java:4593)
        at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
        at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
        at org.apache.hadoop.hbase.thrift.CallQueue$Call.run(CallQueue.java:64)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: hconnection-0x69eeaa65 closed
        at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegion(ConnectionManager.java:1146)
        at org.apache.hadoop.hbase.client.RpcRetryingCallerWithReadReplicas.getRegionLocations(RpcRetryingCallerWithReadReplicas.java:300)
        ... 23 more


检索半天依然没有找到有用的信息。先做一次记录。

yibei8811
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 2
    评论
专栏目录
Hive调用MapReduce任务失败
sky_flying1的博客
09-24 4899
1. Hive简介Hive（蜂巢）Apache Hadoop生态圈的构成之一，其主要功能是基于Hadoop提供MapReduce的类SQL查询。Hive的语法规则和Mysql中SQL的语法规则极为相似，有Mysql使用经验的同学都能轻松上手。 Hive的执行分为本地模式和集群模式。本地模式执行简单的Hive SQL，不需要启动MapReduce程序，如-- 本地模式 select col_name
MapReduce运行机制(二) 失败
ThisIsNobody的博客
06-26 894
任务/application master/NM/RM任务运行失败1) 任务代码异常        &gt; JVM在退出前向application master发送错误报告，报告被记录用户日志        &gt; application master将任务标记为failed，释放容器和资源        &gt; 如果是Streaming任务，如果Streaming进程以非零代码退出，则标记...
探讨MapReduce失败的几种情形
andyguan01_2的博客
04-22 3461
在现实情况中，程序运行可能遇到各种故障：用户代码错误不断、进程崩溃、机器故障等等。使用Hadoop最主要的好处之一是它能处理此类故障并让你能够成功完成作业。五门需要考虑以下实体的失败：任务、application master、节点管理器和资源管理器。 1、任务运行失败 首先考虑任务失败的情况。最常见的情况是map任务或reduce任务中的用户代码抛出运行异常。如果发生这种情况，任务JVM会在退出...
Hadoop项目实战之从HBase中读取数据
说文科技，做有态度的研究。
06-30 5055
MapReduce实战之从HBase中读取数据 一.环境配置 1.本次实验的主要配置环境如下： 物理机：windows 10 虚拟机：VMware pro 12，用其分别创建了三个虚拟机，其ip地址分别为192.168.211.3 hadoop2.6.4 Server version: 5.7.21 MySQL Community Server (GPL) 二.需求分析 1.从H...
hive12启动报错org.apache.thrift.server.TThreadPoolServer.<init>(Lorg/apache/thrift/server/TThreadPoolSer...
weixin_34209406的博客
06-27 662
执行如下命令启动hive服务：./bin/hive --service hiveserver，报如下错误：Starting Hive Thrift ServerException in thread "main" java.lang.NoSuchMethodError: org.apache.thrift.server.TThreadPoolServer.&lt;init&gt;(Lorg/apa...
Java/Web调用Hadoop进行MapReduce示例代码
08-28
Java/Web调用Hadoop进行MapReduce示例代码 本篇文章主要介绍了Java/Web调用Hadoop进行MapReduce示例代码，旨在帮助开发者快速了解如何将Hadoop集成到Java@Web应用程序中。本文涵盖了Hadoop环境搭建、Java/Web工程...
Hadoop中MapReduce基本案例及代码（五）
01-07
每一个键值对调用一次map函数。 写自己的逻辑，对输入的key，value处理，转换成新的key,value输出。 对输出的key,value进行分区。 对相同分区的数据，按照key进行排序（默认按照字典排序）、分组。相同key的value...
Hadoop-mapreduce过程.doc
06-26
MapReduce是Apache Hadoop的核心组件之一，它为大规模数据处理提供了分布式计算框架。本文将从客户端、JobTracker、TaskTracker和Child四个角度，详细阐述MapReduce的工作流程。 1. **客户端提交任务** MapReduce...
大数据Hadoop核心之MapReduce详解
01-17
MapReduce是Hadoop核心模块之一，作为一个分布式运算程序的编程框架，用于用户开发基于Hadoop的数据分析应用。MapReduce的核心功能是将用户编写的业务逻辑代码和自带默认组件整合成一个完整的分布式运算程序，并发...
Hadoop-MapReduce-学习资料及文档.zip
最新发布
10-02
内容概要： mr执行笔记； mapreduce框架的规范； wc流程.xls； wordcount的伪代码； yarn提交job的源码流程； YARN中提交job的详细流程； 打开流的关键代码； 打开流的调用流程； 日志格式；
hadoop执行mapreduce过程reduce不执行原因
u013013024的博客
09-23 7307
1.如果你的map过程中没有context.write()是不执行reduce过程的； 2.如果你的map过程中context.write()的map后的的部分数据出现问题，不符合reduce接受的数据也会不执行reduce 比如说你的日志文件中有一个空行是不符合reduce的接受数据reduce不执行；
在hadoop运行MapReduce失败原因及其解决方法
热门推荐
胖头鱼青年的博客
11-17 1万+
在hadoop运行MapReduce失败原因及其解决方法 刚开始接触在hadoop集群上运行MapReduce，但由于自己能力有限，一开始运行时遇见了各种各样的bug，最终靠重装hadoop解决了所有问题。 本文便是对之前遇见的各种各样bug进行一个总结 错误一： 在输入完指令： hadoop jar original-wordcount-1.0-SNAPSHOT.jar remove.TestWordCount 出现 错误提示：拒绝连接 原因： 可能是因为之前运行的时候不小心打开了slave1
MapReduce之如何处理失败的task
happy19870612's blog
11-18 4425
一 常见容错场景分析 1.1作业某个任务阻塞了，长时间占用资源不释放 1.2在MapTask任务运行完毕，ReduceTask运行过程中，某个MapTask节点挂了，或者某个MapTask结果存放的那磁盘坏掉了   二 作业某个任务阻塞了，长时间占用资源不释放 这种问题通常是由于程序bug，数据特性造成的，会让程序阻塞，任务运行停滞不前。在我们表面上看来是任务停滞不前。   这种问题
Hadoop HA HDFS启动错误之org.apache.hadoop.ipc.Client: Retrying connect to server问题解决
weixin_33949359的博客
03-19 1813
近日，在搭建Hadoop HA QJM集群的时候，出现一个问题，如本文标题。网上有很多HA的博文，其实比较好的博文就是官方文档，讲的已经非常详细。所以，HA的搭建这里不再赘述。本文就想给出一篇org.apache.hadoop.ipc.Client: Retrying connect to server错误的解决的方法。因为在搜索引擎中输入了错误问题，没有找到一篇解决问题的。...
hadoop namenode启动报Permission denied
shouwangV6的专栏
11-14 4070
昨天需要重启一下集群，集群是chd版的，重启后发现namenode启动不了，查看日志一直报这个错 2015-11-13 21:56:14,538 INFO org.apache.hadoop.http.HttpServer2: Jetty bound to port 50070 2015-11-13 21:56:14,538 INFO org.mortbay.log: jetty-6.1.26
利用java api读取hbase数据遇到的一些坑及解决方法
weixin_38425162的博客
02-17 2647
报错信息： can not resolve hostname 解决方法：去C:\Windows\System32\drivers\etc\hosts 里配置一下这个hostname和对应的IP java.lang.RuntimeException: org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after at...
深入探索Hadoop与MapReduce集群框架
- 大数据课程介绍：Hadoop是大数据处理的基础课程之一，通常会涵盖大数据的基本概念、Hadoop的架构和应用。 - 学习建议：学习Hadoop时，建议从基础理论开始，理解分布式系统的原理，然后逐步掌握Hadoop的组件及其...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值