hive自定义函数分为三类:
UDF(User-Defined-Function)普通函数, 一进一出
UDAF(User-Defined Aggregation Function)聚合函数,多进一出
UDTF(User-Defined Table-Generating Functions)表生成函数, 一进多出
UDF简单案例
- 依赖
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.udf</groupId>
<artifactId>udf</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-common</artifactId>
<version>2.7.3</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>2.2</version>
<executions>
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<filters>
<filter>
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
- 编写类,集成 UDF,重写 evaluate 函数
package com.udf;
import org.apache.commons.lang.StringUtils;
import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;
public class udf_zdy extends UDF {
public Text evaluate(Text str){
if (str== null)
return null;
if (StringUtils.isBlank(str.toString()))
return null;
return new Text(str.toString().toUpperCase()+"自定义UDF函数");
}
}
3,进入hive ,添加 jar 包到hive 的环境中
hive> add jar /home/udf-1.0-SNAPSHOT.jar;
4, 创建临时函数
hive> create temporary function toupp as "com.udf.udf_zdy";
//temporary表示为临时方法,当会话结束后失效;udffunc为hive中定义的函数名,‘hive.udf.UDFFunc’为自定义方法的全类路径
5,查询使用
hive> select toupp(name) from a;
OK
A自定义UDF函数
B自定义UDF函数
C自定义UDF函数
D自定义UDF函数
Y自定义UDF函数
U自定义UDF函数
Time taken: 0.682 seconds, Fetched: 6 row(s)
6,相关常用语句
-- 查看函数列表
hive> show functions;
--删除临时函数
语法:DROP TEMPORARY FUNCTION [IF EXISTS] function_name;
创建永久函数的语法:
CREATE FUNCTION [DB_NAME.]FUNCTION_NAME AS CLASS_NAME
[USING JAR|FILE|ARCHIVE 'file_uri' [, JAR|FILE|ARCHIVE 'file_uri'] ];
file_uri:是hdfs上的jar包目录
--创建永久函数的语法:
CREATE FUNCTION [DB_NAME.]FUNCTION_NAME AS CLASS_NAME
[USING JAR|FILE|ARCHIVE 'file_uri' [, JAR|FILE|ARCHIVE 'file_uri'] ];
file_uri:是hdfs上的jar包目录