Docker实战（十五）：Docker安装Hive环境

最新推荐文章于 2024-08-21 10:45:05 发布

golden_lion

最新推荐文章于 2024-08-21 10:45:05 发布

阅读量1.9k

点赞数

分类专栏： Docker

Docker 专栏收录该内容

17 篇文章 0 订阅

订阅专栏

Docker安装Hive环境

Hive安装

<code class="hljs avrasm has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># Hive必须运行在Hadoop之上，则需要先安装Hadoop环境，而且还需要MySQL数据库，具体Hadoop安装请参考上一篇文章，我们这里继承上一篇已经安装好的Hadoop镜像</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 下载Hive</span>
$ wget http://apache<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.mirrors</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.ionfish</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.org</span>/hive/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/apache-hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>-bin<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tar</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.gz</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 解压Hive压缩包</span>
$ tar -zxvf apache-hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>-bin<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tar</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.gz</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 下载MySQL驱动包</span>
$ wget http://cdn<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.mysql</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.com</span>//Downloads/Connector-J/mysql-connector-java-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.38</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tar</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.gz</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 解压MySQL驱动压缩包</span>
$ tar -zxvf mysql-connector-java-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.38</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.tar</span><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;">.gz</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 需要修改下面Hive相关的配置文件</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li></ul>

Dockerfile文件

<code class="hljs coffeescript has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">##</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># version : birdben/hive:v1</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># desc : 当前版本安装的hive</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">######</span><span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">##</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 设置继承自我们创建的 hadoop 镜像</span>
FROM birdben/<span class="hljs-attribute" style="box-sizing: border-box; color: rgb(0, 136, 0);">hadoop</span>:v1

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 下面是一些创建者的基本信息</span>
MAINTAINER birdben (<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">191654006</span>@<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">163.</span>com)

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 设置环境变量，所有操作都是非交互式的</span>
ENV DEBIAN_FRONTEND noninteractive

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 添加 supervisord 的配置文件，并复制配置文件到对应目录下面。（supervisord.conf文件和Dockerfile文件在同一路径）</span>
COPY supervisord.conf /etc/supervisor/conf.d/supervisord.conf

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 复制 hive-1.2.1 文件到镜像中（hive-1.2.1 文件夹要和Dockerfile文件在同一路径），这里直接把上一篇Hadoop环境直接用上了</span>
ADD apache-hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>-bin /software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 复制MySQL的驱动包到镜像中</span>
COPY mysql-connector-java-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.38</span>/mysql-connector-java-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.38</span>-bin.jar /software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/lib/mysql-connector-java-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5.1</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.38</span>-bin.jar

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 授权HIVE_HOME路径给admin用户</span>
RUN sudo chown -R admin /software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"># 执行supervisord来同时执行多个命令，使用 supervisord 的可执行路径启动服务。</span>
CMD [<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"/usr/bin/supervisord"</span>]</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li></ul>

Dockerfile源文件链接：

https://github.com/birdben/birdDocker/blob/master/hive/Dockerfile

supervisor配置文件内容

<code class="hljs vala has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 配置文件包含目录和进程</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 第一段 supervsord 配置软件本身，使用 nodaemon 参数来运行。</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 第二段包含要控制的 2 个服务。每一段包含一个服务的目录和启动这个服务的命令。</span>

[supervisord]
nodaemon=<span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span>

[program:sshd]
command=/usr/sbin/sshd -D

[program:metastore]
command=/software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin/hive --service metastore

[program:hiveserver2]
command=/software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin/hive --service hiveserver2
</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li></ul>

配置HIVE_HOME/conf/hive-env.sh（默认不存在，将hive-env.sh.template复制并改名为hive-env.sh）

<code class="hljs fix has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-attribute" style="box-sizing: border-box;">HADOOP_HOME</span>=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">/software/hadoop-2.7.1</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

配置HIVE_HOME/conf/hive-log4j.properties（默认不存在，将hive-log4j.properties.template复制并改名为hive-log4j.properties）

配置HIVE_HOME/conf/hdfs-site.xml（默认不存在，将hive-default.xml.template复制并改名为hive-site.xml）

<code class="hljs xml has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-pi" style="color: rgb(0, 102, 102); box-sizing: border-box;"><?xml version="1.0" encoding="UTF-8" standalone="no"?></span>
<span class="hljs-pi" style="color: rgb(0, 102, 102); box-sizing: border-box;"><?xml-stylesheet type="text/xsl" href="configuration.xsl"?></span>

<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- metastore我的mysql不是在该server上，是在另一台Docker镜像中 --></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>hive.metastore.local<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>false<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
  <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- mysql服务的ip和端口号 --></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>javax.jdo.option.ConnectionURL<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>jdbc:mysql://172.17.0.2:3306/hive<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>javax.jdo.option.ConnectionDriveName<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>com.mysql.jdbc.Driver<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>javax.jdo.option.ConnectionUserName<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>root<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>javax.jdo.option.ConnectionPassword<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>root<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- hive的仓库目录，需要在HDFS上创建，并修改权限 --></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>hive.metastore.warehouse.dir<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
        <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>/hive/warehouse<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
    <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;"><!-- 运行hive得主机地址及端口，即本机ip和端口号，启动metastore服务 --></span>
      <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>hive.metastore.uris<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">name</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"><<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>thrift://Ben:9083<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">value</span>></span>
    <span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">property</span>></span>
<span class="hljs-tag" style="color: rgb(0, 102, 102); box-sizing: border-box;"></<span class="hljs-title" style="box-sizing: border-box; color: rgb(0, 0, 136);">configuration</span>></span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li></ul>

控制台终端

<code class="hljs vala has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 先启动MySQL的Docker容器</span>
$ sudo docker run -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9999</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3306</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3306</span> -t -i -v /docker/mysql/data:/software/mysql-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5.6</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.22</span>/data birdben/mysql:v1

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 然后查看MySQL的Docker容器的IP地址，Hive连接MySQL的IP地址需要配置这个IP</span>
$ sudo docker inspect --format <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'{{.NetworkSettings.IPAddress}}'</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">34</span>b5ac61b8bd
$ <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172.17</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.2</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 构建镜像</span>
$ docker build -t=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"birdben/hive:v1"</span> .
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 执行已经构件好的镜像，挂载在宿主机器的存储路径也不同，-h设置hostname，Hadoop配置文件需要使用</span>
$ docker run -h Ben -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9998</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">22</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9083</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9083</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9000</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9000</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8088</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8088</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50020</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50020</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50070</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50070</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50075</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50075</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50090</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">50090</span> -t -i <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'birdben/hive:v1'</span>


<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 然后直接通过ssh远程连接使用admin账号远程登录Hive的Docker容器</span>
$ ssh root@<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10.211</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.55</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span> -p <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9998</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 启动hive shell，发现如下的错误，原因是还没有启动Hadoop，所以hive连接Hadoop时，报错java.net.ConnectException: Call From Ben/172.17.0.3 to Ben:9000 failed on connection exception: java.net.ConnectException: Connection refused;</span>

$ admin@Ben:/$ cd /software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin
$ admin@Ben:/$ ./hive shell
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">16</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">25</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">48</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">40</span><span class="hljs-constant" style="box-sizing: border-box;"> WARN </span>conf.HiveConf: HiveConf of name hive.metastore.local does not exist

Logging initialized <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">using</span> configuration in file:/software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/conf/hive-log4j.properties
Exception in thread <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"main"</span> java.lang.RuntimeException: java.net.ConnectException: Call From Ben/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">172.17</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.0</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.3</span> to Ben:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9000</span> failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http:<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">//wiki.apache.org/hadoop/ConnectionRefused</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 所以我们先启动Hadoop</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 先初始化namenode</span>
$ admin@Ben:/$ cd /software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin
$ admin@Ben:/software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/sbin$ ./hdfs namenode -format

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 启动所有服务</span>
admin@Ben:/$ cd /software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/sbin
admin@Ben:/software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/sbin$ ./start-all.sh

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 执行jps命令之前需要先把$JAVA_HOME/bin添加到PATH环境变量</span>
admin@Ben:/$ export PATH=/software/jdk7/bin/:$PATH

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 执行jps命令如果能看到下面的6个进程就说明Hadoop启动的没有问题</span>
admin@Ben:/software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/sbin$ jps
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">985</span> Jps
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">282</span> DataNode
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">587</span> ResourceManager
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">436</span> SecondaryNameNode
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">166</span> NameNode
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">691</span> NodeManager

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 此时重新启动hive shell，就可以成功登录hive了</span>
$ admin@Ben:/$ cd /software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin
$ admin@Ben:/$ ./hive shell
hive>
hive> show databases;
OK
<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">default</span>
Time taken: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.323</span> seconds, Fetched: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> row(s)

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 注意：这里我是使用了之前的MySQL的Docker镜像，因为之前的MySQL的Docker镜像已经处理了root账号的更改密码和远程登录授权问题，所以这里没有涉及这些问题，具体设置可以参考之前的Docker安装MySQL镜像的文章</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 我们需要预先在mysql中创建一个hive的数据库，因为hive-site.xml是连接到这个hive数据库的，所有的hive元数据都是存在这个hive数据库中的</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 我们在hive中创建新的数据库和表来验证hive的元数据都存储在mysql了</span>

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 在hive中创建一个新的数据库test_hive，test_hive这个数据库会对应mysql中的hive数据库中的DBS表中的一条记录</span>
hive><span class="hljs-constant" style="box-sizing: border-box;"> CREATE </span>DATABASE test_hive;

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 在hive中创建一个新的表test_person，test_person这个表会对应mysql中的hive数据库中的TBLS表中的一条记录</span>
hive><span class="hljs-constant" style="box-sizing: border-box;"> USE </span>test_hive;
hive><span class="hljs-constant" style="box-sizing: border-box;"> CREATE </span>TABLE test_person (id INT,name <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">string</span>)<span class="hljs-constant" style="box-sizing: border-box;"> ROW </span>FORMAT<span class="hljs-constant" style="box-sizing: border-box;"> DELIMITED </span>FIELDS<span class="hljs-constant" style="box-sizing: border-box;"> TERMINATED </span>BY <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'\t'</span>;


<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 在hive创建表的时候可能会遇到如下问题，是因为MySQL数据库字符集设置的utf-8导致的</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># Specified key was too long; max key length is 767 bytes</span>
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 修改MySQL的hive数据库的字符集为latin1就好用了</span>
$ alter database hive character <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">set</span> latin1;
<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 参考：http://blog.163.com/zhangjie_0303/blog/static/990827062013112623615941/</span> 

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># test_person.txt</span>
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>   John
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>   Ben
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>   Allen
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>   Jimmy
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>   Will
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>   Jackson

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 用scp命令将本地test_person.txt文件传到hive的Docker容器中</span>
$ scp -P <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9998</span> /Users/ben/workspace_git/birdDocker/hive/test_person.txt admin@<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10.211</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.55</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.4</span>:/software/hive-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.2</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 导入数据到test_person.txt到test_person表</span>
hive><span class="hljs-constant" style="box-sizing: border-box;"> LOAD </span>DATA<span class="hljs-constant" style="box-sizing: border-box;"> LOCAL </span>INPATH <span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'/software/hive-1.2.1/test_person.txt'</span><span class="hljs-constant" style="box-sizing: border-box;"> OVERWRITE </span>INTO<span class="hljs-constant" style="box-sizing: border-box;"> TABLE </span>test_person;
Loading data to table test_hive.test_person
Table test_hive.test_person stats: [numFiles=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, numRows=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>, totalSize=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">45</span>, rawDataSize=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>]
OK
Time taken: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.885</span> seconds

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 查看test_person表数据</span>
hive> select * from test_person;
OK
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>   John
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>   Ben
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>   Allen
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>   Jimmy
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>   Will
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>   Jackson
Time taken: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0.7</span> seconds, Fetched: <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span> row(s)

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 查看test_hive数据库在HDFS中存储的目录</span>
$ admin@Ben:$ cd /software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 查看HDFS中/hive/warehouse目录下的所有文件，此目录是在hive-site.xml中hive.metastore.warehouse.dir参数配置的路径/hive/warehouse</span>
$ admin@Ben:/software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin$ ./hdfs dfs -ls /hive/warehouse/
Found <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> items
drwxr-xr-x   - admin supergroup          <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2016</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">25</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">39</span> /hive/warehouse/test_hive.db

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 查看test_person表在HDFS中存储的目录</span>
$ admin@Ben:/software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin$ ./hdfs dfs -ls /hive/warehouse/test_hive.db/
Found <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> items
drwxr-xr-x   - admin supergroup          <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2016</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">25</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">52</span> /hive/warehouse/test_hive.db/test_person

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 在深入一层就能看到我们导入的文件test_person.txt了</span>
$ admin@Ben:/software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin$ ./hdfs dfs -ls /hive/warehouse/test_hive.db/test_person/
Found <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> items
-rwxr-xr-x   <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> admin supergroup         <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">45</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2016</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">06</span>-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">25</span> <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11</span>:<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">52</span> /hive/warehouse/test_hive.db/test_person/test_person.txt

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># 查看test_person.txt文件里的内容，就是我们导入的内容</span>
$ admin@Ben:/software/hadoop-<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2.7</span><span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">.1</span>/bin$ ./hdfs dfs -cat /hive/warehouse/test_hive.db/test_person/test_person.txt
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>   John
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>   Ben
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>   Allen
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>   Jimmy
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>   Will
<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>   Jackson

<span class="hljs-preprocessor" style="color: rgb(68, 68, 68); box-sizing: border-box;"># OK，大功告成了，可以回家了 ^_^</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li><li style="box-sizing: border-box; padding: 0px 5px;">93</li><li style="box-sizing: border-box; padding: 0px 5px;">94</li><li style="box-sizing: border-box; padding: 0px 5px;">95</li><li style="box-sizing: border-box; padding: 0px 5px;">96</li><li style="box-sizing: border-box; padding: 0px 5px;">97</li><li style="box-sizing: border-box; padding: 0px 5px;">98</li><li style="box-sizing: border-box; padding: 0px 5px;">99</li><li style="box-sizing: border-box; padding: 0px 5px;">100</li><li style="box-sizing: border-box; padding: 0px 5px;">101</li><li style="box-sizing: border-box; padding: 0px 5px;">102</li><li style="box-sizing: border-box; padding: 0px 5px;">103</li><li style="box-sizing: border-box; padding: 0px 5px;">104</li><li style="box-sizing: border-box; padding: 0px 5px;">105</li><li style="box-sizing: border-box; padding: 0px 5px;">106</li><li style="box-sizing: border-box; padding: 0px 5px;">107</li><li style="box-sizing: border-box; padding: 0px 5px;">108</li><li style="box-sizing: border-box; padding: 0px 5px;">109</li><li style="box-sizing: border-box; padding: 0px 5px;">110</li><li style="box-sizing: border-box; padding: 0px 5px;">111</li><li style="box-sizing: border-box; padding: 0px 5px;">112</li><li style="box-sizing: border-box; padding: 0px 5px;">113</li><li style="box-sizing: border-box; padding: 0px 5px;">114</li><li style="box-sizing: border-box; padding: 0px 5px;">115</li><li style="box-sizing: border-box; padding: 0px 5px;">116</li><li style="box-sizing: border-box; padding: 0px 5px;">117</li><li style="box-sizing: border-box; padding: 0px 5px;">118</li><li style="box-sizing: border-box; padding: 0px 5px;">119</li><li style="box-sizing: border-box; padding: 0px 5px;">120</li><li style="box-sizing: border-box; padding: 0px 5px;">121</li><li style="box-sizing: border-box; padding: 0px 5px;">122</li><li style="box-sizing: border-box; padding: 0px 5px;">123</li><li style="box-sizing: border-box; padding: 0px 5px;">124</li><li style="box-sizing: border-box; padding: 0px 5px;">125</li><li style="box-sizing: border-box; padding: 0px 5px;">126</li><li style="box-sizing: border-box; padding: 0px 5px;">127</li><li style="box-sizing: border-box; padding: 0px 5px;">128</li><li style="box-sizing: border-box; padding: 0px 5px;">129</li><li style="box-sizing: border-box; padding: 0px 5px;">130</li><li style="box-sizing: border-box; padding: 0px 5px;">131</li></ul><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li><li style="box-sizing: border-box; padding: 0px 5px;">24</li><li style="box-sizing: border-box; padding: 0px 5px;">25</li><li style="box-sizing: border-box; padding: 0px 5px;">26</li><li style="box-sizing: border-box; padding: 0px 5px;">27</li><li style="box-sizing: border-box; padding: 0px 5px;">28</li><li style="box-sizing: border-box; padding: 0px 5px;">29</li><li style="box-sizing: border-box; padding: 0px 5px;">30</li><li style="box-sizing: border-box; padding: 0px 5px;">31</li><li style="box-sizing: border-box; padding: 0px 5px;">32</li><li style="box-sizing: border-box; padding: 0px 5px;">33</li><li style="box-sizing: border-box; padding: 0px 5px;">34</li><li style="box-sizing: border-box; padding: 0px 5px;">35</li><li style="box-sizing: border-box; padding: 0px 5px;">36</li><li style="box-sizing: border-box; padding: 0px 5px;">37</li><li style="box-sizing: border-box; padding: 0px 5px;">38</li><li style="box-sizing: border-box; padding: 0px 5px;">39</li><li style="box-sizing: border-box; padding: 0px 5px;">40</li><li style="box-sizing: border-box; padding: 0px 5px;">41</li><li style="box-sizing: border-box; padding: 0px 5px;">42</li><li style="box-sizing: border-box; padding: 0px 5px;">43</li><li style="box-sizing: border-box; padding: 0px 5px;">44</li><li style="box-sizing: border-box; padding: 0px 5px;">45</li><li style="box-sizing: border-box; padding: 0px 5px;">46</li><li style="box-sizing: border-box; padding: 0px 5px;">47</li><li style="box-sizing: border-box; padding: 0px 5px;">48</li><li style="box-sizing: border-box; padding: 0px 5px;">49</li><li style="box-sizing: border-box; padding: 0px 5px;">50</li><li style="box-sizing: border-box; padding: 0px 5px;">51</li><li style="box-sizing: border-box; padding: 0px 5px;">52</li><li style="box-sizing: border-box; padding: 0px 5px;">53</li><li style="box-sizing: border-box; padding: 0px 5px;">54</li><li style="box-sizing: border-box; padding: 0px 5px;">55</li><li style="box-sizing: border-box; padding: 0px 5px;">56</li><li style="box-sizing: border-box; padding: 0px 5px;">57</li><li style="box-sizing: border-box; padding: 0px 5px;">58</li><li style="box-sizing: border-box; padding: 0px 5px;">59</li><li style="box-sizing: border-box; padding: 0px 5px;">60</li><li style="box-sizing: border-box; padding: 0px 5px;">61</li><li style="box-sizing: border-box; padding: 0px 5px;">62</li><li style="box-sizing: border-box; padding: 0px 5px;">63</li><li style="box-sizing: border-box; padding: 0px 5px;">64</li><li style="box-sizing: border-box; padding: 0px 5px;">65</li><li style="box-sizing: border-box; padding: 0px 5px;">66</li><li style="box-sizing: border-box; padding: 0px 5px;">67</li><li style="box-sizing: border-box; padding: 0px 5px;">68</li><li style="box-sizing: border-box; padding: 0px 5px;">69</li><li style="box-sizing: border-box; padding: 0px 5px;">70</li><li style="box-sizing: border-box; padding: 0px 5px;">71</li><li style="box-sizing: border-box; padding: 0px 5px;">72</li><li style="box-sizing: border-box; padding: 0px 5px;">73</li><li style="box-sizing: border-box; padding: 0px 5px;">74</li><li style="box-sizing: border-box; padding: 0px 5px;">75</li><li style="box-sizing: border-box; padding: 0px 5px;">76</li><li style="box-sizing: border-box; padding: 0px 5px;">77</li><li style="box-sizing: border-box; padding: 0px 5px;">78</li><li style="box-sizing: border-box; padding: 0px 5px;">79</li><li style="box-sizing: border-box; padding: 0px 5px;">80</li><li style="box-sizing: border-box; padding: 0px 5px;">81</li><li style="box-sizing: border-box; padding: 0px 5px;">82</li><li style="box-sizing: border-box; padding: 0px 5px;">83</li><li style="box-sizing: border-box; padding: 0px 5px;">84</li><li style="box-sizing: border-box; padding: 0px 5px;">85</li><li style="box-sizing: border-box; padding: 0px 5px;">86</li><li style="box-sizing: border-box; padding: 0px 5px;">87</li><li style="box-sizing: border-box; padding: 0px 5px;">88</li><li style="box-sizing: border-box; padding: 0px 5px;">89</li><li style="box-sizing: border-box; padding: 0px 5px;">90</li><li style="box-sizing: border-box; padding: 0px 5px;">91</li><li style="box-sizing: border-box; padding: 0px 5px;">92</li><li style="box-sizing: border-box; padding: 0px 5px;">93</li><li style="box-sizing: border-box; padding: 0px 5px;">94</li><li style="box-sizing: border-box; padding: 0px 5px;">95</li><li style="box-sizing: border-box; padding: 0px 5px;">96</li><li style="box-sizing: border-box; padding: 0px 5px;">97</li><li style="box-sizing: border-box; padding: 0px 5px;">98</li><li style="box-sizing: border-box; padding: 0px 5px;">99</li><li style="box-sizing: border-box; padding: 0px 5px;">100</li><li style="box-sizing: border-box; padding: 0px 5px;">101</li><li style="box-sizing: border-box; padding: 0px 5px;">102</li><li style="box-sizing: border-box; padding: 0px 5px;">103</li><li style="box-sizing: border-box; padding: 0px 5px;">104</li><li style="box-sizing: border-box; padding: 0px 5px;">105</li><li style="box-sizing: border-box; padding: 0px 5px;">106</li><li style="box-sizing: border-box; padding: 0px 5px;">107</li><li style="box-sizing: border-box; padding: 0px 5px;">108</li><li style="box-sizing: border-box; padding: 0px 5px;">109</li><li style="box-sizing: border-box; padding: 0px 5px;">110</li><li style="box-sizing: border-box; padding: 0px 5px;">111</li><li style="box-sizing: border-box; padding: 0px 5px;">112</li><li style="box-sizing: border-box; padding: 0px 5px;">113</li><li style="box-sizing: border-box; padding: 0px 5px;">114</li><li style="box-sizing: border-box; padding: 0px 5px;">115</li><li style="box-sizing: border-box; padding: 0px 5px;">116</li><li style="box-sizing: border-box; padding: 0px 5px;">117</li><li style="box-sizing: border-box; padding: 0px 5px;">118</li><li style="box-sizing: border-box; padding: 0px 5px;">119</li><li style="box-sizing: border-box; padding: 0px 5px;">120</li><li style="box-sizing: border-box; padding: 0px 5px;">121</li><li style="box-sizing: border-box; padding: 0px 5px;">122</li><li style="box-sizing: border-box; padding: 0px 5px;">123</li><li style="box-sizing: border-box; padding: 0px 5px;">124</li><li style="box-sizing: border-box; padding: 0px 5px;">125</li><li style="box-sizing: border-box; padding: 0px 5px;">126</li><li style="box-sizing: border-box; padding: 0px 5px;">127</li><li style="box-sizing: border-box; padding: 0px 5px;">128</li><li style="box-sizing: border-box; padding: 0px 5px;">129</li><li style="box-sizing: border-box; padding: 0px 5px;">130</li><li style="box-sizing: border-box; padding: 0px 5px;">131</li></ul>

test_hive数据库在MySQL的hive数据库中的DBS表中的一条记录

DBS表

test_person表在MySQL的hive数据库中的TBLS表中的一条记录

TBLS表

元数据解析

在MySQL可以查看Hive元数据表，除了DBS和TBLS存储数据库和表的基本信息，其他表的说明见下表:

MySQL的表	说明
BUCKETING_COLS	Hive表CLUSTERED BY字段信息(字段名，字段序号)
CDS
COLUMNS_V2	存放表格的字段信息
DATABASE_PARAMS
DBS	存放hive所有数据库信息
FUNCS
FUNC_RU
GLOBAL_PRIVS
PARTITIONS	Hive表分区信息(创建时间，具体的分区)
PARTITION_KEYS	Hive分区表分区键(名称，类型，comment，序号)
PARTITION_KEY_VALS	Hive表分区名(键值，序号)
PARTITION_PARAMS
PART_COL_STATS
ROLES
SDS	所有hive表、表分区所对应的hdfs数据目录和数据格式。
SD_PARAMS
SEQUENCE_TABLE	SEQUENCE_TABLE表保存了hive对象的下一个可用ID，如’org.apache.hadoop.hive.metastore.model.MTable’, 21，则下一个新创建的hive表其TBL_ID就是21，同时SEQUENCE_TABLE表中271786被更新为26(这里每次都是+5)。同样，COLUMN，PARTITION等都有相应的记录
SERDES	Hive表序列化反序列化使用的类库信息
SERDE_PARAMS	序列化反序列化信息，如行分隔符、列分隔符、NULL的表示字符等
SKEWED_COL_NAMES
SKEWED_COL_VALUE_LOC_MAP
SKEWED_STRING_LIST
SKEWED_STRING_LIST_VALUES
SKEWED_VALUES
SORT_COLS	Hive表SORTED BY字段信息(字段名，sort类型，字段序号)
TABLE_PARAMS	表级属性，如是否外部表，表注释等
TAB_COL_STATS
TBLS	所有hive表的基本信息
VERSION