全是代码,cv替换成你们自己的就行。涉及到业务,中间删改了的,可能有点问题,自己看着改把。
spark操作:
drop table test.base_tb;
CREATE TABLE IF NOT EXISTS test.base_tb(
pk_id BIGINT COMMENT '自增id',
datatime STRING COMMENT '数据日期'
.....
)
row format delimited fields terminated by '\t';
LOAD DATA INPATH '/hdfs/xlsx2hdfs/hdfs文件' INTO TABLE test.base_tb;
select * from test.base_tb limit 1;
//spark做计算查询 /查询的数据创建一个临时视图
select name,idcord,province,city,idcord_loc,mar_status,home_address from test.base_tb;
-- 创建一个临时表或视图
DROP VIEW IF EXISTS temp_result;
CREATE OR REPLACE TEMPORARY VIEW temp_result AS
SELECT name, idcord, province, city, idcord_loc, mar_status, home_address
FROM test.base_tb;
//临时表保存到本地。路径:/ProgramData/MySQL/MySQL Server 8.0/Uploads/
select * from temp_result limit 1;
mysql操作:
通过使用 @variable 来指定变量,然后在 SET 子句中将这些变量映射到相应的列。
LOAD DATA INFILE '/ProgramData/MySQL/MySQL Server 8.0/Uploads/hmkjtest_base_assetpackage_tb.csv'
INTO TABLE package.case_person_info
FIELDS TERMINATED BY ',' -- 指定字段分隔符
LINES TERMINATED BY '\n' -- 指定行分隔符
IGNORE 1 LINES
(@name, @idcord, @province, @city, @idcord_loc, @mar_status, @home_address)
SET
case_per_name = @name,
card_id = @idcord,
case_per_pro_no = @province,
case_per_city_no = @city,
case_house_address = @idcord_loc,
marital_status = @mar_status,
case_per_family_address = @home_address;
select * from package.case_person_info limit 1;
简单讲就是创建临时表或者视图,把需要的字段映射进mysql。中间细节有点忘了,行不通的话,想问的评论或者私信问我,有时间就回。