Sparksql读取hive回顾

数据准备
hive这边建表

create table employee (
employee_id int,
employee_name string,
manager_id int
)
comment 'this is employee table'     
row format delimited             
fields terminated by '\t'       
lines terminated by '\n'        
stored as textfile;  

create table employeesalary (
employee_id int,
salary int,
datetime date
)
comment 'this is employeesalary table'     
row format delimited             
fields terminated by '\t'       
lines terminated by '\n'        
stored as textfile;

注意这个日期
mysql的date是 ‘2021-08-08’,就是date就是要带单引号
hive的date是2021-08-08,至于datestime是有小时分钟秒的

导入数据
就是在非hive界面,随便创建一个文档,把数据写入到文档里面

vim t1

001	Tom	null
002	Alex	001
003 Sophia	001
004	Wiliam	002
005	Amelia	null
vim t2
001	10000	2022-01-20
001	10010	2022-02-20
002	10005	2022-01-20
002	10000	2022-02-20

hive这边

load data local inpath 'data/t1' into table employee;
load data local inpath 'data/t2' into table employeesalary;

spark的开发
创建一个工程,添加scala的环境,然后把hive-site.xml添加到resource里面
hive-site.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
--><configuration>

<!--配置mysql的连接字符串-->
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://qianfeng03:3306/hive?createDatabaseIfNotExist=true&amp;
  • 1
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值