(已解决)hive insert 插入报错[Tez session start thread] tez.TezSessionState: Failed to start Tez session

 上图为问题现象    求大佬救命啊  ,拜托拜托      

现在是,不用tez引擎就没事,一切正常,切换tez引擎,就是查询正常,创建正常,就是不能插入

应该可以排除资源不足,公司服务器,剩余可用内存46G

此问题以解决,其实没啥问题,就是环境关机,开机,都重新启动后,由于之前的人,没有对数据库后端的nfs做永久挂载,我重新挂上之后,可能现在这种情况,使用tez的话,需要重新做初始化,把mysql数据库的hive表删了重新初始化,就解决了,但是还有个小问题,说表数据格式啥之类的,但是不影响使用,插入数据是执行成功了。

下面是日志输出信息

 以下是我的配置信息

这里是,以为是内存小,调大了一点,原来是1024   1cpu   0.4

tez-site.xml

 yarn-site.xml

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>
<!-- Site specific YARN configuration properties -->
<!-- 指定MR走shuffle -->
  <property>
    <name>yarn.nodemanager.aux-services</name>
    <value>mapreduce_shuffle</value>
  </property>

 <!--Configurations for the state-store of ResourceManager-->
     <property>
       <name>yarn.resourcemanager.recovery.enabled</name>
       <value>true</value>
     </property>
     <property>
       <name>yarn.resourcemanager.store.class</name>
       <value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateS
tore</value>
     </property>
     <property>
       <name>yarn.resourcemanager.zk-address</name>
       <value>hadoop01:2181,hadoop02:2181,hadoop03:2181</value>
       <description>ZooKeeper服务的地址,多个地址使用逗号隔开</description>
     </property>
     <!--Configurations for HA of ResourceManager-->
"~/module/hadoop-3.1.3/etc/hadoop/yarn-site.xml" 350L, 11747C

  <!-- 哪些用户有权向队列提交作业 -->
  <property>
    <name>yarn.scheduler.capacity.root.olap.acl_submit_applications</name>
    <value>hadoop</value>
  </property>

  <!-- 哪些用户有权操作队列,管理员权限(查看/杀死) -->
  <property>
    <name>yarn.scheduler.capacity.root.olap.acl_administer_queue</name>
    <value>hadoop</value>
  </property>

  <!-- 哪些用户有权提交任务优先级 -->
  <property>
    <name>yarn.scheduler.capacity.root.olap.acl_application_max_priority</name>
    <value>hadoop</value>
  </property>

  <!-- 任务的超时时间设置:yarn application -appId appId -updateLifetime Timeou
t -->

  <!-- 如果application指定了超时时间,则提交到该队列的application能够指定的最大
超时时间不能超过该值 -->
  <property>
    <name>yarn.scheduler.capacity.root.olap.maximum-application-lifetime</name>
    <value>-1</value>
  </property>

  <!-- 如果application没有指定超时时间,则用default-application-lifetime作为默>
认值 -->
  <property>
    <name>yarn.scheduler.capacity.root.olap.default-application-lifetime</name>
    <value>-1</value>
  </property>


</configuration>

hive-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <!-- jdbc连接URL -->
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://10.230.248.217:3306/hive_metastore?useSSL=false&amp;use
Unicode=true&amp;characterEncoding=UTF-8</value>
  </property>

  <!-- jdbc连接的Driver -->
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>eezero</value>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>Cmcc1qaz@WSX</value>
  </property>

  <!-- Hive元数据存储版本的验证 -->
  <property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
  </property>

  <!-- 元数据存储授权 -->
  <property>
    <name>hive.metastore.event.db.notification.api.auth</name>
    <value>false</value>
  </property>

"~/module/hive-3.1.2/conf/hive-site.xml" 129L, 3445C
  <property>
    <name>hive.support.concurrency</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.exec.dynamic.partition.mode</name>
    <value>nonstrict</value>
  </property>
  <property>
    <name>hive.txn.manager</name>
    <value>org.apache.hadoop.hive.ql.lockmgr.DbTxnManager</value>
  </property>
  <property>
    <name>hive.enforce.bucketing</name>
    <value>true</value>
  </property>
  <property>
    <name>hive.exec.dynamic.partition.mode</name>
    <value>nonstrict</value>
  </property>
  <property>
    <name>hive.compactor.worker.threads</name>
    <value>4</value>
  </property>
  <property>
    <name>hive.compactor.initiator.on</name>
    <value>on</value>
  </property>
  <property>
    <name>hive.execution.engine</name>
    <value>tez</value>
  </property>
  <property>
    <name>hive.tez.container.size</name>
    <value>1024</value>
  </property>

</configuration>

求大佬救命,拜托啦           

  • 3
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
这个错误是在运行Hive时出现的,报错信息为"FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask"。这通常是由于内存不足导致的问题。根据引用中的解决方案,可以尝试以下两种方法来解决这个问题。 第一种方法是关闭虚拟内存检查。你可以修改yarn-site.xml文件,将yarn.nodemanager.vmem-check-enabled的值设置为false。具体操作如下: ```bash cd /opt/module/hadoop-3.1.3/etc/hadoop vim yarn-site.xml <property> <name>yarn.nodemanager.vmem-check-enabled</name> <value>false</value> </property> ``` 第二种方法是在mapred-site.xml中设置Map和Reduce任务的内存配置。你可以根据自己的机器内存大小和应用情况修改以下配置: ```bash <property> <name>mapreduce.map.memory.mb</name> <value>1536</value> </property> <property> <name>mapreduce.map.java.opts</name> <value>-Xmx1024M</value> </property> <property> <name>mapreduce.reduce.memory.mb</name> <value>3072</value> </property> <property> <name>mapreduce.reduce.java.opts</name> <value>-Xmx2560M</value> </property> ``` 通过尝试这两种方法,你应该能够解决"FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask"的问题。希望对你有帮助。<span class="em">1</span><span class="em">2</span><span class="em">3</span> #### 引用[.reference_title] - *1* [已解决FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask](https://blog.csdn.net/qq_43408367/article/details/128601584)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"] - *2* *3* [FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask](https://blog.csdn.net/weixin_44870066/article/details/123902421)[target="_blank" data-report-click={"spm":"1018.2226.3001.9630","extra":{"utm_source":"vip_chatgpt_common_search_pc_result","utm_medium":"distribute.pc_search_result.none-task-cask-2~all~insert_cask~default-1-null.142^v93^chatsearchT3_1"}}] [.reference_item style="max-width: 50%"] [ .reference_list ]

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值