需求:hive表email_status_result中有个邮箱(receiver )字段,查询结果需要脱敏展示,表结构如下
CREATE TABLE email_status_result(
receiver STRING,
email_status STRING,
status_count INT
)COMMENT 'email_status aggregation data'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
STORED AS TEXTFILE;
未脱敏情况下查询 select receiver from email_status_result limit 10
解决方案:
UDF分为三种,分别如下
1、UDF(User-Defined-Function),一进一出(输入一行,输出一行),比如:lowser()等。
2、UDAF(User-Defined Aggregation Funcation),多进一出(输入多行,输出一行),比如:avg()等。
3、UDTF(User-Defined Table-Generating Functions),一进多出(输入一行,输出多行),比如:collect_set()
该脱敏需求符合输入一行,输出一行,因此我们采取UDF实现
1、代码实现
public class EncrptEmailUdf extends UDF {
private String getEncrpEmail(String email) {
String user = email.split("@")[0];
String newEmail = "";
if (user.length() > 1) {
newEmail = String.format("%s%s%s", user.substring(0, 1),
DigestUtils.md5Hex(email),
user.substring(user.length() - 1));
}
return newEmail;
}
public String evaluate(String email) {
return getEncrpEmail(email);
}
}
2、通过eclipse等工具导出encrptEmail.jar并上传encrptEmail.jar到客户端服务器/opt/目录下
3、进入hive客户端执行
hive> add jar /opt/encrptEmail.jar
4、创建临时函数
hive> create temporary function encrptEmail as 'com.sohu.sc.tool.EncrptEmailUdf';
5、脱敏查询
hive>select encrptEmail(receiver) from email_status_result limit 10