Hive数仓基本

【hive建表】

create table CharacterLogin(
AppID string,
GameID string,
ChildId string,
IP string,
ServerID string,
AccountID string,
CharacterID string,
LogType string,
LogTime int,
PlatformChannelId string,
IsLogin int,
OnlineTime int,
Level int,
VIPLevel int
)
partitioned by(year string,month string,day string) row format delimited fields terminated by '|';

【数据导入】
将数据根据时间分区导入Hive，分区格式为，年/月/日（将文件用xshell上传到linux过程略）

shell脚本如下

#!/bin/bash

path="/home/li/Desktop/CharacterLogin"
 
files=$(ls $path)
 
for filename in $files
do 
  filename=${filename%.*}
     
    var1=`echo "$filename"|awk -F '-' '{print $1}'`
    var2=`echo "$filename"|awk -F '-' '{print $2}'`  
    var3=`echo "$filename"|awk -F '-' '{print $3}'`      
  $HIVE_HOME/bin