执行shell脚本批量删除hive表分区
执行命令: sh delete_partitions.sh schema.table_name 2017-11-01 2017-12-01
delete_partitions.sh脚本如下:
#!/bin/sh
if [ $# == 3 ]; then
begin_date=`date -d "+0 day $2" +%Y-%m-%d`
end_date=`date -d "+0 day $3" +%Y-%m-%d`
date=${end_date}
sql=''
while [[ "${date}" > "${begin_date}" || "${date}" = "${begin_date}" ]]
do
echo $date
sql=${sql}"ALTER TABLE $1 DROP IF EXISTS PARTITION(date = '$date');"
echo ${sql}
date=`date -d "$date -1 days" +"%Y-%m-%d"`
done
echo "hive -e '${sql}' "
hive -e "${sql}"
elif [ $# == 1 ]; then
date=`date -d -1days '+%Y-%m-%d'`
echo "hive -e 'ALTER TABLE $1 DROP IF EXISTS PARTITION(date = '$date');'"
hive -e "ALTER TABLE $1 DROP IF EXISTS PARTITION(date = '$date');"
else
echo 'Parameter error!'
fi
在后面的版本里发现删除日期分区这样就行了:
alter table schema.table_name drop partition (ds<'2018-08-01');
如果是spark server的执行引擎怎加上以下参数:
set spark.syntax.extension=true;