spark连接hive

最新推荐文章于 2023-07-20 21:12:44 发布

于鲲LaNce

最新推荐文章于 2023-07-20 21:12:44 发布

阅读量261

点赞数

分类专栏： spark 文章标签： hive spark mysql IDEA

本文链接：https://blog.csdn.net/YK_LaNce/article/details/96145302

版权

本文介绍了如何配置Spark连接Hive，包括复制和修改配置文件，如hive-site.xml和hdfs-site.xml，设置环境变量，解决权限问题，并在IDEA中配置Spark项目以支持Hive。测试代码阶段，确保Hadoop版本匹配，有时可能需要重启才能运行。

摘要由CSDN通过智能技术生成

一、配置文件

先将hive/conf下的hive-site.xml复制到spark/conf下
对照下面的hive-site.xml修改

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,