Tez
Hive底层的计算引擎
微电子学与固体电子学-俞驰
如切如磋,如琢如磨,臻于至善。
展开
-
Hive引擎改为Tez笔记
环境:组件 版本 Hadoop 3.1.2 Hive 3.0.0 Tez 0.9.2 #-------------------------------------------------https://mirrors.bfsu.edu.cn/apache/tez/0.9.2/apache-tez-0.9.2-src.tar.gzcdapache-tez-0.9.2-src/tez-dist/mvn package -Dhadoop.version=3.原创 2020-05-31 23:42:35 · 1511 阅读 · 0 评论 -
Tez UI界面一直处于loading
按下F12得到:可以看出是跨域问题导致的,Access to XMLHttpRequest at 'http://desktop:8088/ws/v1/cluster/apps/application_1591242817964_0009?_=1591253071174' from origin 'http://desktop:8880' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' heade...原创 2020-06-04 14:49:54 · 1103 阅读 · 1 评论 -
Hive on Tez出现exec.Task: Failed to execute tez graph. java.lang.NullPointerException
报错复现:hive>select count(*) from student;报错如下:2020-06-03 22:00:36,787 ERROR [57ee4918-ac03-4f15-82c0-0cfd7cbcda73 main] exec.Task: Failed to execute tez graph.java.lang.NullPointerException at org.apache.hadoop.hive.ql.exec.tez.TezSessionState.ge原创 2020-06-03 22:14:48 · 6737 阅读 · 0 评论 -
Cannot set priority of timelineserver process 23314以及leveldb-timeline-store.ldb does not exist解决
./yarn-daemons.sh start timelineserver报错如下:Desktop: ERROR: Cannot set priority of timelineserver process 23314原因:这是因为你已经启动了一个timelineserver的进程,关闭该进程即可。解决方案:ps -ef |grep timelineserverkill -9 timelineserver的pid...原创 2020-06-03 20:46:33 · 1583 阅读 · 0 评论 -
Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster
Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster$HADOOP_HOME/etc/hadoop下面的tez-site.xml下面的tez.lib.uris.classpath整个property删除或者注释掉即可。原创 2020-06-03 19:37:40 · 4350 阅读 · 4 评论 -
Illegal processing instruction target (xml)
java.lang.RuntimeException: com.ctc.wstx.exc.WstxParsingException: Illegal processing instruction target ("xml"); xml (case insensitive) is reserved by the specs.at [row,col,system-id]: [19,5,"file:/home/appleyuchi/bigdata/hadoop-3.1.2/etc/hadoop/tez-sit.原创 2020-06-03 14:29:04 · 2495 阅读 · 0 评论 -
Exception in thread main java.lang.NoClassDefFoundError: org/apache/tez/dag/api/TezConfiguration
hive on tez,为了这个配置今天是第四天了,MLGB的!!!###################################################################################################这个报错是在启动hive客户端的时候触发了。######################################下面是解决方案#####################################直接说解决方案:~原创 2020-06-03 08:59:07 · 7163 阅读 · 10 评论 -
[ERROR] bower xxx解决方案
编译tez的时候出现如下报错:[ERROR] bower ember-cli-shims#0.0.6 invalid-meta for:/tmp/appleyuchi/bower/1b3f6982d79d1ec37bf564c2aa592dce-9260-cAoRa4/bower.json[ERROR] bower ember-cli-shims#0.0.6 invalid-meta The "main" field has to contain only...原创 2020-06-01 17:52:47 · 392 阅读 · 0 评论 -
cannot find symbol [ERROR] symbol: class DistributedFileSystem
编译tez时,由于使用的是hadoop3.1.2所以出现以下错误,[ERROR] COMPILATION ERROR :[INFO] -------------------------------------------------------------[ERROR] /home/appleyuchi/bigdata/apache-tez-0.9.2-src/tez-api/src/test/java/org/apache/tez/client/TestTezClientUtils.java.转载 2020-06-01 17:25:19 · 1397 阅读 · 0 评论 -
org.apache.maven.plugin.MojoExecutionException: protoc version is libprotoc 3.6.1, expected versio
在编译tez的时候碰到这个问题:ubuntu19.10的系统,apt 默认安装3.6.10的protocwget https://github.com/google/protobuf/releases/download/v2.5.0/protobuf-2.5.0.tar.gztar -xzf protobuf-2.5.0.tar.gzcd protobuf-2.5.0/./autogen.sh./configuremakesudo make installsudo subl /et原创 2020-06-01 17:13:24 · 1248 阅读 · 0 评论 -
tez安装官方文档整理+翻译
Install/Deploy Instructions for TezReplace x.y.z with the tez release number that you are using. E.g. 0.5.0. For Tez versions 0.8.3 and higher, Tez needs Apache Hadoop to be of version 2.6.0 or higher. For Tez version 0.9.0 and higher, Tez needs Apache H翻译 2020-06-01 16:05:20 · 875 阅读 · 0 评论 -
Failing because I am unlikely to write too排查方案
tez本身不带logs,我在输入hive的时候碰到了下面的报错Logging initialized using configuration in file:/home/appleyuchi/bigdata/apache-hive-3.0.0-bin/conf/hive-log4j2.properties Async: trueException in thread "main" java.lang.RuntimeException: java.io.IOException: Previous wr原创 2020-05-31 23:21:50 · 3945 阅读 · 2 评论