drop table if exists v_byronli_ind.dual_test;
create table v_byronli_ind.dual_test(ecif_no string,name string,code string)
row format delimited fields terminated by '\t'
lines terminated by '\n'
load date local inpath 'file://mnt/test.txt' overwrite into table v_byronli_ind.dual_test
如果文件导入后查询出来为中文乱码,需要先将文件转换其编码格式为utf-8
其他笔记(pyspark环境下操作python)
from pyspark import SparkContext
from pyspark.sql import SparkSession,SQLContext
from pyspark.sql import Row,functions as F
from pyspark.sql.functions import datediff,months_between
from pandas import DataFrame
import pandas as pd
spark.sql('''代码''')#sql语句(注意读取出来的数据需要转化用到to_Pandas())