pinot_引入apache pinot 0 5 0

pinot

We are excited to announce that Apache Pinot 0.5.0 has been released in September 2020. Apache Pinot is a real-time distributed datastore, designed to answer OLAP queries with low latency. In this release, we continue to add new query and ingestion capacity and at the same time make Pinot more performant, easy to manage and reliable. This post highlights a few features in this release. The full release notes can be found here.

我们很高兴地宣布，Apache Pinot 0.5.0已于2020年9月发布。Apache Pinot是一种实时分布式数据存储，旨在以低延迟回答OLAP查询。 在此版本中，我们继续增加新的查询和接收功能，同时使Pinot性能更高，易于管理和可靠。 这篇文章重点介绍了此版本中的一些功能。 完整的发行说明可以在这里找到。

新的Cluster Manager UI和查询控制台 (New Cluster Manager UI and Query Console)

Pinot 0.5.0 replaces the existing consoles with an app developed using React + TypeScript + Material UI. The new consoles have a modern Material UI styling (see samples below). There are more status pages added to the cluster manager UI including table details page, segment details pages, zookeeper data exploration page, query response stats pages and so on. Hope you will enjoy the new looks of Pinot!

Pinot 0.5.0用使用React + TypeScript + Material UI开发的应用程序替换了现有的控制台。 新的控制台具有现代的Material UI样式(请参见下面的示例)。 集群管理器UI中添加了更多状态页面，包括表详细信息页面，段详细信息页面，zookeeper数据浏览页面，查询响应统计信息页面等。 希望您会喜欢黑皮诺的新外观！

New Cluster Manager UI

新的Cluster Manager UI

New Query Console

新查询控制台

Pinot查询中的Groovy转换UDF(Groovy Transform UDF in Pinot Query)

Backed in 0.4.0, Pinot added the support for transformation functions during ingestion using Apache Groovy, which allowed users to define derived fields directly in the schema. In 0.5.0, the Groovy transformation functions are also added in Pinot queries. Pinot queries now are able to run a Groovy script (e.g., string concatenation) in queries as a UDF. For example, you can now run the following Pinot query to concatenate the return values of columnA and columnB:

从0.4.0开始，Pinot使用Apache Groovy添加了在摄取期间对转换功能的支持，该功能允许用户直接在模式中定义派生字段。 在0.5.0中，还可以在Pinot查询中添加Groovy转换函数。 Pinot查询现在可以作为UDF在查询中运行Groovy脚本(例如，字符串连接)。 例如，您现在可以运行以下Pinot查询来连接columnA和columnB的返回值：

SELECT GROOVY(// Function result type
‘{“returnType”: “STRING”, “isSingleValue”: true}’, // Function definition
‘arg0 + “ “ + arg1’,// Arguments
columnA, columnB) FROM myTable

This feature makes transformation of query results much easier and it is all done by Pinot directly.

此功能使查询结果的转换变得更加容易，并且全部由Pinot直接完成。

提取期间的JSON对象转换 (JSON Object Conversion During Ingestion)

Before 0.5.0, map type complex data in Pinot are converted to string. However, the format of the stored string is not parsable using common jsonExtract functions. Consequently, the fields in the complex object can not be retrieved easily.

在0.5.0之前，Pinot中的地图类型复杂数据将转换为字符串。 但是，使用通用的jsonExtract函数无法解析存储的字符串的格式。 因此，复杂对象中的字段无法轻松检索。

Pinot 0.5.0 rewrites the JSON objects conversion during ingestion. With that, complex map objects with nested structure are converted into proper JSON strings. More important, these stored JSON objects can later be queried using jsonExtractScalar() functions. For example, now you can add the following to your PQL query:

Pinot 0.5.0会在摄取期间重写JSON对象转换。 这样，具有嵌套结构的复杂地图对象将转换为正确的JSON字符串。 更重要的是，这些存储的JSON对象可以稍后使用jsonExtractScalar()函数进行查询。 例如，现在您可以将以下内容添加到您的PQL查询中：

SELECT JSONEXTRACTSCALAR(profile_json_str, // jsonField‘$.name’,        // jsonPath to extract the scalar‘STRING’         // ‘resultType’) FROM myTable

The complete list of JSON functions and its syntax can be found here.

JSON函数及其语法的完整列表可以在此处找到。

支持JDBC客户端 (Support for JDBC clients)

Pinot 0.5.0 continues to expand its supports for popular client applications and input formats. It adds support for JDBC driver so that native Java JDBC clients can execute queries on Pinot. Please check out this doc for examples and the complete documentations.

Pinot 0.5.0继续扩展了对流行的客户端应用程序和输入格式的支持。 它增加了对JDBC驱动程序的支持，以便本机Java JDBC客户端可以在Pinot上执行查询。 请查看此文档以获取示例和完整的文档。

支持协议缓冲区输入格式 (Support for Protocol Buffer Input Format)

Pinot 0.5.0 continues to expand its supports for popular input formats. It now supports to ingestion of data inputs in the popular Proto-buf format.

Pinot 0.5.0继续扩展了对流行输入格式的支持。 现在，它支持以流行的Proto-buf格式提取数据输入。

在分段加载期间允许创建星型树 (Allow Star-Tree Creation During Segment Load)

Star-Tree index is an index for speeding up aggregation queries (e.g., SUM, COUNT and so on). It provides excellent query latency improvement with moderate space overhead. Previously if one wants to add star-tree indexes to an existing table, he/she has to re-create the entire table. Pinot 0.5 allows users to modify table configs to add Star-Tree index and then reload the table segments to add the indexes on the fly.

Star-Tree索引是用于加快聚合查询(例如SUM，COUNT等)的索引。 它以适度的空间开销提供了出色的查询延迟改进。 以前，如果要向现有表添加星型树索引，则他/她必须重新创建整个表。 Pinot 0.5允许用户修改表配置以添加Star-Tree索引，然后重新加载表段以动态添加索引。

添加对黑皮诺查询的地理支持 (Add Geo Support for Pinot Queries)

Many business operations need to query geo spatial data. Pinot 0.5.0 adds native supports for geo-spatial data models including geometry and geography. Pinot users now can add common geo relationship function like ST_CONTAINS and ST_EQUALS to their Pinot queries. The complete list of supported function can be found in this PR. This is the beginning of a bigger project to provide a complete and efficient solution for Geo data query in Pinot. Stay tuned!

许多业务操作需要查询地理空间数据。 Pinot 0.5.0增加了对地理空间数据模型(包括几何和地理)的原生支持。 Pinot用户现在可以在其Pinot查询中添加常见的地理关系功能，例如ST_CONTAINS和ST_EQUALS 。 可以在此PR中找到受支持功能的完整列表。 这是一个更大的项目的开始，该项目旨在为Pinot中的地理数据查询提供完整而有效的解决方案。 敬请关注！

Download page: https://pinot.apache.org/download/Getting started: https://docs.pinot.apache.org/getting-started

下载页面： https : //pinot.apache.org/download/入门： https : //docs.pinot.apache.org/getting-started

Slack channel: https://communityinviter.com/apps/apache-pinot/apache-pinot

松弛频道： https ： //communityinviter.com/apps/apache-pinot/apache-pinot

特别感谢 (Special thanks)

We would like to take a moment to thank the Pinot community for supporting our Product. We have seen a healthy increase of commits to Pinot code base since September 2019 (see the graph below). Here we also like to thank the committers and contributors for their contribution to this release.

我们想花一点时间感谢黑皮诺社区对我们产品的支持。 自2019年9月以来，我们看到对Pinot代码库的提交呈健康增长(请参见下图)。 在这里，我们还要感谢提交者和贡献者对此版本所做的贡献。

Source) 来源)

翻译自: https://medium.com/apache-pinot-developer-blog/introducing-apache-pinot-0-5-0-e27983b6a678

pinot