Cloudera HDP 3 对 Hive 的改进

最新推荐文章于 2024-07-24 17:25:05 发布

houzhizhen

最新推荐文章于 2024-07-24 17:25:05 发布

阅读量2k

点赞数

分类专栏： hive 文章标签： hive

本文链接：https://blog.csdn.net/houzhizhen/article/details/124337136

版权

hive 专栏收录该内容

154 篇文章 15 订阅

订阅专栏

HDP 从3.0 开始使用 hive-3.1.0。

HDP 的 3 系列有3.0 和 3.1 两个大版本。3.0 有 3.0.0 和 3.0.1 两个小版本。3.1 有3.1.0，3.1.4，3.1.5 三个小版本（截止2022-04-22）.

3.1.5 新特性

参考HDP-3.1.5 new features

	
Core Capabilities

Support single catalog for Hive and Spark using the Hive Metastore (HMS) translation layer.

bug fixes

合入了大量 hive-4.0 的patch。
其中导致结果不一致的 bug 有12个。
HDP-3.1.5 fixed_issues

Patch 列表

3.1.4 新特性

参考HDP-3.1.4 new features

Core Capabilities

Automatic partition management synchronizes changes in the metadata and on the file system.
You can now configure how long you can retain partition data and metadata.
Hive Warehouse Connector now validates mapping of columns against those in Hive to alert the user to input errors.
Writing a DataFrame to Hive supports specifying partitions.
A new MergeBuilder interface for HiveWarehouseSession API operations supports merging tables.
Upgrade

HiveStrictManagedMigration has two new options:
Specify the number of threads for processing tables in parallel. Default: The number of CPU cores.

Specify the the type of tables to process. For example, MANAGED_TABLE, EXTERNAL_TABLE. Default: All tables.

bug fixes

其中导致结果不一致的 bug 有8个。
HDP-3.1.4 fixed_issues
Patch 列表

3.1.0 新特性

Hive 没有新特性，参考HDP-3.1.0 new features

bug fixes

其中导致结果不一致的 bug 有10个。
HDP-3.1.0 fixed_issues
Patch 列表

3.0.1 新特性

没有新特性，参考HDP-3.0.1 new features

bug fixes

其中导致结果不一致的 bug 有 2 个。
HDP-3.0.1 fixed_issues
Patch 列表

3.0.0

新特性

参考HDP-3.0.0 new features

Core Capabilities

Workload management for LLAP. You can now run LLAP, in a multi-tenant environment without worrying about resource competition.

ACID v2 and ACID on by default. ACID v2 has performance improvements in both storage format and execution engine, there is either equal or better performance when compared to non-ACID tables. ACID on is enabled by default to allow full support for data updates.

Materialized view navigation. Hive’s query engine now supports materialized view. The query engine will automatically use materialized view when they are available to speed up your queries.

Information schema. Hive now exposes the metadata of the database (tables, columns etc.) via Hive SQL interface directly.Integration

Hive Warehouse Connector for Spark. Hive Warehouse Connector allows you to connect Spark application with Hive data warehouses. The connector automatically handles ACID tables.
JDBC storage connector. You can now map any JDBC database’s tables into Hive and query those tables in conjunction with other tables