Hive：数据仓库利器

大袤宏图

已于 2024-03-17 23:22:46 修改

阅读量331

点赞数 2

分类专栏：大数据文章标签：数据仓库 hive hadoop

于 2024-03-16 15:23:05 首次发布

本文链接：https://blog.csdn.net/mao1994aa/article/details/136763404

版权

6 篇文章 0 订阅

订阅专栏

1. 简介

Hive是一个基于Hadoop的开源数据仓库工具，可以用来存储、查询和分析大规模数据。Hive使用SQL-like的HiveQL语言来查询数据，并将其结果存储在Hadoop的文件系统中。

介绍 Hive 的核心概念，例如表、分区、桶、HQL 等。

组成	详情
UI	The user interface for users to submit queries and other operations to the system. As of 2011 the system had a command line interface and a web based GUI was being developed.
Driver	The component which receives the queries. This component implements the notion of session handles and provides execute and fetch APIs modeled on JDBC/ODBC interfaces.
Compiler	The component that parses the query, does semantic analysis on the different query blocks and query expressions and eventually generates an execution plan with the help of the table and partition metadata looked up from the metastore.
Metastore	The component that stores all the structure information of the various tables and partitions in the warehouse including column and column type information, the serializers and deserializers necessary to read and write data and the corresponding HDFS files where the data is stored.
Execution Engine	The component which executes the execution plan created by the compiler. The plan is a DAG of stages. The execution engine manages the dependencies between these different stages of the plan and executes these stages on the appropriate system components.

类型	详情
Tables	These are analogous to Tables in Relational Databases. Tables can be filtered, projected, joined and unioned. Additionally all the data of a table is stored in a directory in HDFS. Hive also supports the notion of external tables wherein a table can be created on prexisting files or directories in HDFS by providing the appropriate location to the table creation DDL. The rows in a table are organized into typed columns similar to Relational Databases.
Partitions	Each Table can have one or more partition keys which determine how the data is stored, for example a table T with a date partition column ds had files with data for a particular date stored in the <table location>/ds=<date> directory in HDFS. Partitions allow the system to prune data to be inspected based on query predicates, for example a query that is interested in rows from T that satisfy the predicate T.ds = '2008-09-01' would only have to look at files in <table location>/ds=2008-09-01/ directory in HDFS.
Buckets	Data in each partition may in turn be divided into Buckets based on the hash of a column in the table. Each bucket is stored as a file in the partition directory. Bucketing allows the system to efficiently evaluate queries that depend on a sample of data (these are queries that use the SAMPLE clause on the table).