hive on tez java.lang.NumberFormatException: For input string: “30s“

已于 2022-09-02 11:20:19 修改
阅读量995 收藏 1
点赞数
文章标签： hive
于 2022-09-02 11:10:45 首次发布
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/weixin_38751513/article/details/126658704
版权 
2022-09-02 11:01:03,983 [INFO] [main] |app.DAGAppMaster|: Creating DAGAppMaster for applicationId=application_1658852014961_0068, attemptNum=1, AMContainerId=container_1658852014961_0068_01_000001, jvmPid=8152, userFromEnv=ucmed, cliSessionOption=true, pwd=/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001, localDirs=/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068, logDirs=/usr/local/src/hadoop/hadoop-3.3.1/logs/userlogs/application_1658852014961_0068/container_1658852014961_0068_01_000001
2022-09-02 11:01:04,417 [INFO] [main] |app.DAGAppMaster|: Created DAGAppMaster for application appattempt_1658852014961_0068_000001, versionInfo=[ component=tez-dag, version=0.9.2, revision=10cb3519bd34389210e6511a2ba291b52dcda081, SCM-URL=scm:git:https://gitbox.apache.org/repos/asf/tez.git, buildTime=2019-03-19T20:51:15Z ]
2022-09-02 11:01:04,457 [INFO] [main] |app.DAGAppMaster|: 
/************************************************************
[system properties]
os.name: Linux
os.version: 3.10.0-514.el7.x86_64
java.home: /usr/local/src/jdk/jdk1.8.0_311/jre
java.runtime.version: 1.8.0_311-b11
java.vendor: Oracle Corporation
java.version: 1.8.0_311
java.vm.name: Java HotSpot(TM) 64-Bit Server VM
java.class.path: /opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/hive-exec-3.1.2-50b0545693ae9392559584116a172ee32922e57ae9ea0eb7dc120290c217c2d5.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/hadoop-shim-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-api-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-build-tools-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-common-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-runtime-internals-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-runtime-library-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-mapreduce-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-examples-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-dag-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-tests-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-ext-service-tests-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-protobuf-history-plugin-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-yarn-timeline-history-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-yarn-timeline-history-with-acls-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-history-parser-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-job-analyzer-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/tez-javadoc-tools-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/hadoop-shim-2.7-0.9.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/slf4j-api-1.7.10.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-yarn-api-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-lang-2.6.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/guava-11.0.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jsr305-3.0.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-logging-1.1.3.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/protobuf-java-2.5.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-common-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-annotations-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-cli-1.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-math3-3.1.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/xmlenc-0.52.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-codec-1.4.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-io-2.4.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-net-3.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-collections-3.2.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jersey-json-1.9.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jettison-1.3.4.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jaxb-impl-2.2.3-1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jaxb-api-2.2.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/stax-api-1.0-2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/activation-1.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jackson-core-asl-1.9.13.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jackson-mapper-asl-1.9.13.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jackson-jaxrs-1.9.13.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jackson-xc-1.9.13.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/log4j-1.2.17.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-configuration-1.6.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-digester-1.8.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-beanutils-1.7.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-beanutils-core-1.8.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/slf4j-log4j12-1.7.10.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/avro-1.7.4.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/paranamer-2.3.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/snappy-java-1.0.4.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-compress-1.4.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/xz-1.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/gson-2.2.4.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-auth-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/httpclient-4.2.5.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/httpcore-4.2.4.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/apacheds-i18n-2.0.0-M15.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/api-asn1-api-1.0.0-M20.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/api-util-1.0.0-M20.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/zookeeper-3.4.6.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/netty-3.6.2.Final.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/curator-framework-2.7.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/curator-client-2.7.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/curator-recipes-2.7.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/htrace-core-3.1.0-incubating.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/stax-api-1.0.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-yarn-common-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/servlet-api-2.5.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jetty-util-6.1.26.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jersey-core-1.9.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jersey-client-1.9.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/guice-servlet-3.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/guice-3.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/javax.inject-1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/aopalliance-1.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jersey-guice-1.9.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-yarn-client-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/commons-collections4-4.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jetty-6.1.26.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/netty-all-4.0.23.Final.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/xercesImpl-2.9.1.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/xml-apis-1.3.04.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/leveldbjni-all-1.8.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-hdfs-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/RoaringBitmap-0.5.21.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/async-http-client-1.8.16.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-mapreduce-client-core-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-mapreduce-client-common-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-yarn-server-web-proxy-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/hadoop-yarn-server-common-2.7.2.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jetty-server-9.3.24.v20180605.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/javax.servlet-api-3.1.0.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jetty-http-9.3.24.v20180605.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jetty-util-9.3.24.v20180605.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/jetty-io-9.3.24.v20180605.jar:/opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tezlib/lib/metrics-core-3.1.0.jar:
java.io.tmpdir: /opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001/tmp
user.dir: /opt/hdfs/data/nm-local-dir/usercache/ucmed/appcache/application_1658852014961_0068/container_1658852014961_0068_01_000001
user.name: ucmed
************************************************************/
2022-09-02 11:01:04,483 [INFO] [main] |shim.HadoopShimsLoader|: Trying to locate HadoopShimProvider for hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2022-09-02 11:01:04,484 [INFO] [main] |shim.HadoopShimsLoader|: Picked HadoopShim org.apache.tez.hadoop.shim.HadoopShim27, providerName=org.apache.tez.hadoop.shim.HadoopShim25_26_27Provider, overrideProviderViaConfig=null, hadoopVersion=2.7.2, majorVersion=2, minorVersion=7
2022-09-02 11:01:04,508 [INFO] [main] |app.DAGAppMaster|: AM Level configured TaskSchedulers: [0:TezYarn:null],[1:TezUber:null]
2022-09-02 11:01:04,508 [INFO] [main] |app.DAGAppMaster|: AM Level configured ContainerLaunchers: [0:TezYarn:null],[1:TezUber:null]
2022-09-02 11:01:04,508 [INFO] [main] |app.DAGAppMaster|: AM Level configured TaskCommunicators: [0:TezYarn:null],[1:TezUber:null]
2022-09-02 11:01:04,508 [INFO] [main] |app.DAGAppMaster|: Comparing client version with AM version, clientVersion=0.9.2, AMVersion=0.9.2
2022-09-02 11:01:04,602 [INFO] [main] |service.AbstractService|: Service org.apache.tez.dag.app.DAGAppMaster failed in state INITED; cause: java.lang.NumberFormatException: For input string: "30s"
java.lang.NumberFormatException: For input string: "30s"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:589)
	at java.lang.Long.parseLong(Long.java:631)
	at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1311)
	at org.apache.hadoop.hdfs.DFSClient$Conf.<init>(DFSClient.java:502)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:638)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
	at org.apache.tez.common.TezCommonUtils.getTezBaseStagingPath(TezCommonUtils.java:87)
	at org.apache.tez.common.TezCommonUtils.getTezSystemStagingPath(TezCommonUtils.java:146)
	at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:492)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
	at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
2022-09-02 11:01:04,606 [WARN] [main] |service.AbstractService|: When stopping the service org.apache.tez.dag.app.DAGAppMaster : java.lang.NullPointerException
java.lang.NullPointerException
	at org.apache.tez.dag.app.DAGAppMaster.initiateStop(DAGAppMaster.java:2179)
	at org.apache.tez.dag.app.DAGAppMaster.serviceStop(DAGAppMaster.java:2195)
	at org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
	at org.apache.hadoop.service.ServiceOperations.stop(ServiceOperations.java:52)
	at org.apache.hadoop.service.ServiceOperations.stopQuietly(ServiceOperations.java:80)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:171)
	at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
	at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
2022-09-02 11:01:04,606 [ERROR] [main] |app.DAGAppMaster|: Error starting DAGAppMaster
java.lang.NumberFormatException: For input string: "30s"
	at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
	at java.lang.Long.parseLong(Long.java:589)
	at java.lang.Long.parseLong(Long.java:631)
	at org.apache.hadoop.conf.Configuration.getLong(Configuration.java:1311)
	at org.apache.hadoop.hdfs.DFSClient$Conf.<init>(DFSClient.java:502)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:638)
	at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:619)
	at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
	at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2653)
	at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
	at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
	at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
	at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
	at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
	at org.apache.tez.common.TezCommonUtils.getTezBaseStagingPath(TezCommonUtils.java:87)
	at org.apache.tez.common.TezCommonUtils.getTezSystemStagingPath(TezCommonUtils.java:146)
	at org.apache.tez.dag.app.DAGAppMaster.serviceInit(DAGAppMaster.java:492)
	at org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
	at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2662)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
	at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2659)
	at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2464)
2022-09-02 11:01:04,608 [INFO] [Thread-2] |app.DAGAppMaster|: DAGAppMasterShutdownHook invoked

错误日志如图

所用版本为:

hadoop 3.3.1

hive 3.1.2

tez 0.9.2

解决方案如下

在hdfs-site.xml中新增配置

<property>
    <name>dfs.client.datanode-restart.timeout</name>
    <value>30</value>
</property>

问题解决

原因为Hadoop2和Hadoop3的兼容性问题

相关问题如下:

org.apache.ignite.IgniteException: For input string: "30s" in ignite hadoop execution - Stack Overflow

不搬砖的程序员不是好程序员
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeExcepti
01-07
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 解决方法 ...
线上flink任务重启报错(Hadoop问题)java.lang.NumberFormatException: For input string: “30s“
17611119847
01-04 3566
线上Flink任务重启时，提交任务失败，具体报错如下： 2021-12-31 18:09:19 java.lang.NumberFormatException: For input string: "30s" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) at java.lang.Long.parseLong(Long.java:589) at java.lang.Long.parseL
oozie：java.lang.NumberFormatException: For input string: “30s“
IT嫩小白
08-25 1357
oozie启动报错如下： >>> Invoking Main class now >>> Failing Oozie Launcher, IO error org.apache.oozie.action.hadoop.LauncherException: IO error at org.apache.oozie.action.hadoop.LauncherAM.setRecoveryId(LauncherAM.java:487) at org.apache.oozi
oozie 提交任务：java.lang.NumberFormatException: For input string "30s"
shengjk1的博客
03-25 2245
生产环境CDH升级到了 6.3.1，程序提交之后，统统报错了 最终检查之后发现，hadoop 3.0 与之前的版本不兼容。 修改方式： 将现有的hadoop依赖的版本升级到 hadoop 3.0 ...
flink 运维中遇到的问题
h952520296的博客
03-03 8530
1. Java.lang.IllegalStateException: No Executor found. Please make sure to export the HADOOP_CLASSPATH environment variable or have hadoop in your classpath. For more information refer to the "Deployment" section of the official Apache Flink documentation.
java.lang.NumberFormatException: For input string: ““解决方案
热门推荐
娜可璐璐的博客
01-26 10万+
引起异常的主要原因如下： 1.传参字段和映射字段不一致 2.传参类型和映射类型不一致 3.时间类型转换时间戳长度不一致 4.参数长度和数据库不一致 今天在编写代码的时候发现了这样一个问题，平时在将String的字符型数据转化成Iint类型数据的时候，习惯性用的是Integer类中的Integer.parseInt("");方法，但是实际使用的时候却总是报错。 java.lang.NumberFormatException: For input string: “” 报错信息大同小异，大部分都是在上面的语句的
解决启动Azkaban报错问题：java.lang.NoSuchMethodError: com.google.common.collect.ImmutableMap.toImmutableMap
10-15
在Java编程中，`NoSuchMethodError` 是一个常见的运行时异常，它表示在类加载时找到了该类，但在该类中却找不到特定方法的定义。这个问题通常发生在不同版本的库之间存在不兼容的情况，即在编译时使用的库版本包含了...
hive on tez 常见报错问题收集
01-26
可以通过调整`hive.tez.container.size`和`hive.tez.java.opts`来增加内存分配，例如设置前者为9216MB，后者为6144MB。同时，如果物理内存或虚拟内存超出限制，需要调整YARN的配置，如`yarn.scheduler.minimum-...
java.net.SocketException: Connection reset 解决方法
09-05
Java中的`java.net.SocketException: Connection reset`是一个常见的网络编程错误，通常表示在TCP/IP通信过程中，连接突然中断。这个异常可能在客户端或服务器端发生，通常与数据传输的异常中断有关。 首先，我们来...
hadoop java.lang.UnsatisfiedLinkError
09-12
解决方案：Exceptionin thread "main" java.lang.UnsatisfiedLinkError:org.apache.hadoop.util.NativeCrc32.nativeCo
java.lang.NumberFormatException: For input string错误原因
m0_45040545的博客
06-11 5357
1.传参字段和映射字段不一致 2.传参类型和映射类型不一致 3.时间类型转换时间戳长度不一致 4.参数长度和数据库不一致
NumberFormatException: For input string的解决办法
weixin_54232666的博客
01-14 4244
在写接口的时候，出现了NumberFormatException异常，我找到了解决方案。
NumberFormatException: For input string错误解决
Vters的博客
01-13 1万+
For input string的话就是在类型转换的时候出现的问题 String douNum ="3.25" ; int parseInt = Integer.parseInt(douNum); 使用这段代码就会出现这样的错误 Exception in thread "main" java.lang.NumberFormatException: For input string: "3.25" at java.lang.NumberFormatExcepti...
SparkSql读取hive-NumberFormatException:For input string:“0000001_0000“
superMario
08-07 1053
SparkSql读取hive-NumberFormatException:For input string:“0000001_0000” HDP3集群环境下，spark2.3 在使用Spark Sql操作hive表时出现异常: java.lang.NumberFormatException:For input string:“0000001_0000” 出现这个问题的原因其实是 HDP3.0 集成了hive 3.0和 spark 2.3，然而spark却读取不了hive表的数据数据，准确来说是内表的数据。因
Kerberos环境下Oozie使用Shell方式调度Spark程序WordCount
TT-Learning
05-22 390
问题 集群环境：Ambari2.7.3 + HDP3.0.1.0-187 使用Oozie调度，使用jar的时候，有两种jar包，一种是项目打包，包含依赖的jar包，包比较大，一种是项目打包，不包含依赖的jar包，包比较小。 在使用Oozie调度项目jar包时，发现，调度包含依赖的jar包，报错：java.lang.NumberFormatException: For input string: "30s" 在使用Oozie调度项目jar包时，发现，调度不包含依赖的jar包，成功。 这就很难受，那么有没有
java.lang.NumberFormatException: For input string: “22-02-22 07:13:3
qq_57969464的博客
04-18 787
JSON字符串转为Java对象 使用FastJson将json字符串转换为实体对象，对象中包含时间字段时发生错误 java.lang.NumberFormatException: For input string: "22-02-23 07:13:34" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[na:1.8.0_144] at java.lang.Long.parseLon
Flink 《java.lang.NumberFormatException: For input string..数据:》(错误及解决办法 五.）
禧年
12-19 1846
前言： 在学习Flink 过程中，有一段map聚合代码需要把javabean映射出来并返回数据的一个需求。 数据是在当前工程下的resources目录下创建的txt文档。 在执行的时候，报错： java.lang.NumberFormatException: For input string具体数据… 准备解决bug： 此时我内心波涛汹涌，在想为啥会类型错误呢，不过安静下来，根据错误提示进去源码内查看报错说：没有合适的数据类型格式，然后想到了数据类型转换 1. String aLong = String
/root路径下启动hive问题RuntimeException: java.lang.NumberFormatException : For input string: "1s"
u013303361的专栏
05-03 1262
环境：hive0.13.0和Hadoop2.6具体现象：[root@master(3.100) ~]# hiveLogging initialized using configuration in file:/work/hadoop/hive/apache-hive-0.13.1-bin/conf/hive-log4j.propertiesException in thread "main" ja...
hive java.lang.NoClassDefFoundError: org/apache/spark/SparkConf
最新发布
07-27
引用\[1\]:在你提供的引用中，出现了一个Java错误，即"java.lang.NoClassDefFoundError"，这个错误通常是在编译过程中没有问题，但在运行过程中发现缺少类或者有重复类导致的。\[2\]根据错误提示中的关键点"org/apache/hadoop/conf/Configuration"，我们可以看到缺少了这个类。解决这个问题的方法是修改pom.xml文件中的Spark依赖项的作用域(scope)，将其从"provided"改为"compile"。\[3\]这样做可以确保在运行时能够正确加载所需的类。 #### 引用[.reference_title] - *1* *3* [spark 遇到NoClassDefFoundError解决方法： Apache Spark error on start: java.lang.NoClassDefFoundError:...](https://blog.csdn.net/a391000181/article/details/102221192)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^insertT0,239^v3^insert_chatgpt"}} ] [.reference_item] - *2* [问题解决：java运行HiveQL，报错：java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration](https://blog.csdn.net/u013084266/article/details/106805574)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v91^insertT0,239^v3^insert_chatgpt"}} ] [.reference_item] [ .reference_list ]
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值