背景介绍
这篇文章主要说明一下关于Presto中UDF的开发。这个实例中的自定方法的逻辑很简单,只是传如一个字符串,然后在字符串前面拼装一个Hello,只是为了说明如何去自己实现一个Scalar Function。
下面是具体步骤及代码
1.创建Maven工程
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<packaging>presto-plugin</packaging>
<groupId>com.levin.presto.plugin</groupId>
<artifactId>plugin_test</artifactId>
<version>1.0</version>
<parent>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-root</artifactId>
<version>0.187</version>
</parent>
<properties>
<presto.verison>0.187</presto.verison>
</properties>
<dependencies>
<dependency>
<groupId>com.facebook.presto</groupId>
<artifactId>presto-spi</artifactId>
<version>${presto.verison}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.airlift</groupId>
<artifactId>slice</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.airlift</groupId>
<artifactId>units</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.openjdk.jol</groupId>
<artifactId>jol-core</artifactId>
<scope>provided</scope>
</dependency>
</dependencies>
<!-- 屏蔽parent中的plugin 避免打包报错 -->
<build>
<plugins>
<plugin>
<groupId>com.mycila</groupId>
<artifactId>license-maven-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<groupId>pl.project13.maven</groupId>
<artifactId>git-commit-id-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
<plugin>
<groupId>org.gaul</groupId>
<artifactId>modernizer-maven-plugin</artifactId>
<configuration>
<skip>true</skip>
</configuration>
</plugin>
</plugins>
</build>
</project>
HelloFunction.java 用来定义方法实际的逻辑
package com.levin.presto.plugin;
import com.facebook.presto.spi.function.Description;
import com.facebook.presto.spi.function.ScalarFunction;
import com.facebook.presto.spi.function.SqlType;
import com.facebook.presto.spi.type.StandardTypes;
import io.airlift.slice.Slice;
import io.airlift.slice.Slices;
public class HelloFunction {
@Description("Hello Function")
@ScalarFunction("hello") //方法名称
@SqlType(StandardTypes.VARCHAR) //返回类型
public static Slice hello(@SqlType(StandardTypes.VARCHAR) Slice str) {
String value=str.toStringUtf8();
return Slices.utf8Slice("Hello "+value);
}
}
HelloFunctionPlugin.java 用于统一注册到Plugin集合中
package com.levin.presto.plugin;
import com.facebook.presto.spi.Plugin;
import com.google.common.collect.ImmutableSet;
import java.util.Set;
public class HelloFunctionPlugin implements Plugin {
@Override
public Set<Class<?>> getFunctions() {
return ImmutableSet.<Class<?>>builder().add(HelloFunction.class).build();
}
}
mvn 打包命令
mvn clean package -Dcheckstyle.skip=true -Dcheckstyle.skipExec=true
将target下的plugin_test-1.0.jar 复制到 %PRESTO_HOME%/plugin/plugin_test/下,并且复制一个guava-21.0.jar到上面的路径中
PS:plugin_test最好与工程名称一致
Plugins must be installed on all nodes in the Presto cluster (coordinator and workers).
启动Presto然后使用客户端发送查询 select hello(‘levin’)
presto> select hello('levin');
_col0
-------------
Hello levin
(1 row)
Query 20181108_095033_00002_9rg6y, FINISHED, 1 node
Splits: 17 total, 17 done (100.00%)
0:00 [0 rows, 0B] [0 rows/s, 0B/s]
OK 这样就完成了一个自定义的标量函数。
关于聚合函数等其他复杂的函数定义请参考下面的链接。
https://prestodb.io/docs/current/develop/spi-overview.html
https://prestodb.io/docs/current/develop/functions.html
https://github.com/prestodb/presto/tree/master/presto-teradata-functions