kylin（一）介绍编译与单节点测试

最新推荐文章于 2024-08-03 15:22:26 发布

黄土高坡上的独孤前辈

最新推荐文章于 2024-08-03 15:22:26 发布

阅读量1.4k

点赞数

分类专栏： Hive/Kylin数据仓库文章标签： kylin

本文链接：https://blog.csdn.net/lihuazaizheli/article/details/107031041

版权

Hive/Kylin数据仓库专栏收录该内容

24 篇文章 2 订阅

订阅专栏

1. kylin的产生背景

1.1 传统数仓会遇到的问题

	传统的数仓架构，实际上只支持垂直扩展
	Hadoop与BI平台衔接不够成熟，无法提供高效的交互式查询的
	在这个背景下，eBay 2013年 BI on Hadoop

1.2 kylin核心设计理念的诞生的过程

	Hive、Spark SQL等SQL on Hadoop的框架 大规模的并行处理和列式存储
	1亿条记录的  耗时1分钟
	10亿	     ？10
	随着数据的指数级的增长，查询速度是无法保证
	sql：join  group by  sum()  count() count(distinct)
	对于大多数的数据表来说  维度和指标 基本上都是能够去确定的
	预计算的概念 把数据计算完 存起来 ==> 查询
                 join  group by      ==> select ==> 能够节省一些计算

2.kylin的发展历史

	2013年  	ebay启动了BI on Hadoop
	2014年9月	内部版本上线
	2014年10月	正式开源
	2014年11月	apache孵化器项目
	2015年12月	正式成为顶级项目
	2016年3月	kylingence

3.kylin技术架构

3.1 数据源

实时(kafka)/离线(Hadoop、Hive数仓)

3.2 核心模块

(1)cube构建引擎 MR/Spark
(2)元数据管理模块/工具：cube、构建作业等等  存哪里？HBase上
(3)路由模块：
	kylin sql ====> hbase
	问题：
	1.在定义cube 选择维度、度量  很难保证线上的所有查询都会命中我们所设计的cube，对应去查hive里的数据
	2.cube依赖于hive数据表，有可能存在同张hive表 ==> >=2个cube
(4)查询引擎
(5)数据存储：HBase

在这里插入图片描述

4.kylin核心概念

4.1 维度/度量

数据分析当中最最基本的两个概念
(1)维度：审视数据的一个角度，记录数据的一个属性：时间、地点等等
(2)度量：基于数据所计算出来的一个具体的数值
select item,sum(price) from tbl where xxx group by item

4.2 cube/cuboid

cube：n维；对于每一种维度组合，都会去做聚合运算 		                  n种
cuboid：是用来组成cube的

4.3 segment

在kylin当中，设计完cube之后，需要去构建的
每次构建的时候，都会去选择对应的时间范围 1周的数据 segment

5.kylin目录

	[hadoop@hadoop001 apache-kylin-2.6.0]$ ll
	total 48
	drwxr-xr-x.  2 hadoop hadoop 4096 May 17 00:13 bin
	-rw-r--r--.  1 hadoop hadoop  823 Jan 13  2019 commit_SHA1
	drwxr-xr-x.  2 hadoop hadoop 4096 May 16 22:51 conf
	drwxrwxr-x.  2 hadoop hadoop 4096 May  4  2019 ext
	drwxr-xr-x.  2 hadoop hadoop 4096 May  4  2019 lib
	drwxrwxr-x.  2 hadoop hadoop 4096 May 20 04:20 logs
	drwxr-xr-x.  5 hadoop hadoop 4096 Mar 22 09:12 sample_cube
	-rw-rw-r--.  1 hadoop hadoop  242 May 16 23:31 SCSinkTools.json
	drwxr-xr-x. 10 hadoop hadoop 4096 May  4  2019 spark
	drwxrwxr-x.  7 hadoop hadoop 4096 Jan 19 09:37 system_cube
	drwxr-xr-x.  9 hadoop hadoop 4096 May  4  2019 tomcat
	drwxr-xr-x.  2 hadoop hadoop 4096 Jan 13  2019 tool

6.kylin部署架构

	http://kylin.apache.org/docs/install/kylin_cluster.html
	mode：query  用于数据查询的节点	 多台查询节点做负载均衡
		  job    用于构建cube的节点  任务构建引擎高可用

在这里插入图片描述

7.sample_cube体验

	[hadoop@hadoop001 bin]$ ./sample.sh

	Cardinality 维度基数 cout(distinct)之后的结果 item 200

	数据加载
		load table 库名.表名
		Load Table Metadata From Tree
	Aggregation Groups
		聚合组 ==> 是我们做cube优化 维度裁剪的 关键步骤 ==> 对于cube大小、构建速度 起到关键作用的
		Includes  需要包含的维度
		Mandatory Dimensions  强制维度/必须维度 每个查询都会带的维度  一般来说会选时间字段
		Hierarchy Dimensions  层级维度  省市区关系  一/二/三/四/五...类目(电商)
		Joint Dimensions      联合维度  id和name

	Rowkeys
		设计的好与坏  与查询HBase数据的效率 息息相关

	Cube Engine
		MapReduce/Spark

	Advanced Dictionaries
		涉及到精准去重的指标 bitmap/hll(hyperloglog) 有一定的误差，但是存储的空间比小

	Advanced Snapshot Table

	Advanced ColumnFamily
		去重/求和

kylin部署
	[kylin.properties]
	kylin.env.hadoop-conf-dir 指定到具体的$HADOOP_HOME/etc/hadoop

# 动手操作
1.kylin基于cdh5.16.2编译
2.kylin部署好，sample_cube给构建起来 对比一下查Hive和查kylin的速率

8.kylin的编译

 前置准备：
 http://archive.cloudera.com/cdh5/cdh/5/

8.1 nodejs安装

  $> wget https://nodejs.org/dist/v10.9.0/node-v10.9.0-linux-x64.tar.xz
  $> tar xf  node-v10.9.0-linux-x64.tar.xz
  $> ln -s /usr/software/nodejs/bin/npm /usr/local/bin/
  $> ln -s /usr/software/nodejs/bin/node /usr/local/bin/
  $> node -v  ##查看版本

8.2 bower安装

$> npm install -g bower

npm太慢， 淘宝npm镜像使用方法
https://blog.csdn.net/qq_37674858/article/details/80065701

npm --registry https://registry.npm.taobao.org install express

npm install -g bower


ln -s /usr/local/node/node-v6.17.1-linux-x64/bin/bower /usr/local/bin/bower


# 能够执行出命令
[root@hadoop003 /]# bower --allow-root -v
1.8.8

8.3 phantomjs安装

  手动安装部署phantomjs即可，并配置环境变量，下载地址：
  https://github.com/Medium/phantomjs/releases/download/v1.9.19/phantomjs-1.9.8-linux-x86_64.tar.bz2
  
  yum install -y bzip2
  tar -jxvf phantomjs-1.9.8-linux-x86_64.tar.bz2
  
  安装 PhantomJS 安装过程如下
  https://www.cnblogs.com/guotianqi/p/7686441.html
  
  yum -y install wget fontconfig
  
 
    
# 编译报错
  > phantomjs@1.9.20 install /kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs
  > node install.js

Considering PhantomJS found at /usr/local/phantomjs/phantomjs-2.1.1-linux-x86_64/bin/phantomjs
Found PhantomJS at /usr/local/phantomjs/phantomjs-2.1.1-linux-x86_64/bin/phantomjs ...verifying
PhantomJS detected, but wrong version 2.1.1 @ /usr/local/phantomjs/phantomjs-2.1.1-linux-x86_64/bin/phantomjs.
/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/phantomjs is not writable: EACCES: permission denied, mkdir '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/phantomjs'
Download already available at /tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2
Verified checksum of previously downloaded file
Extracting tar contents (via spawned process)
Removing /kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom
Copying extracted folder /tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1593327325826/phantomjs-1.9.8-linux-x86_64 -> /kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom
Phantom installation failed { [Error: EACCES: permission denied, link '/tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1593327325826/phantomjs-1.9.8-linux-x86_64' -> '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom']
  errno: -13,
  code: 'EACCES',
  syscall: 'link',
  path:
   '/tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1593327325826/phantomjs-1.9.8-linux-x86_64',
  dest:
   '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom' } Error: EACCES: permission denied, link '/tmp/phantomjs/phantomjs-1.9.8-linux-x86_64.tar.bz2-extract-1593327325826/phantomjs-1.9.8-linux-x86_64' -> '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/grunt-lib-phantomjs/node_modules/phantomjs/lib/phantom'
npm WARN base@0.0.1 No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@2.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! phantomjs@1.9.20 install: `node install.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the phantomjs@1.9.20 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-06-28T06_55_28_303Z-debug.log

# 报错解决
在Centos上安装phantomjs的过程
https://segmentfault.com/a/1190000013317848
  
/kylin/source/apache-kylin-2.6.0/build/script/build.sh
[root@hadoop003 script]# cat build.sh 
echo 'package front-end'
cd webapp
npm install -g bower                     || { exit 1; }
bower --allow-root install               || { exit 1; }
npm install                                              || { exit 1; }
npm install -g grunt-cli                 || { exit 1; }
grunt dev --buildEnv=dev --buildNumber=`date "+%Y%m%d%H%M%S"`  || { exit 1; }

# 将上面build脚本中的npm install  后面加一个--unsafe-perm
npm clean --force
npm install --unsafe-perm  
npm install --unsafe-perm  -g grunt-cli

8.4 pom.xml文件修改 cdh5.7 ==> 对应所要编译的cdh版本

  $> sed -i "s/2.6.0-cdh5.7.0/2.6.0-cdh5.16.2/g" `grep "cdh" -rl pom.xml`
  $> sed -i "s/1.1.0-cdh5.7.0/1.1.0-cdh5.16.2/g" `grep "cdh" -rl pom.xml`
  $> sed -i "s/1.2.0-cdh5.7.0/1.2.0-cdh5.16.2/g" `grep "cdh" -rl pom.xml`
  $> sed -i "s/3.4.5-cdh5.7.0/3.4.5-cdh5.16.2/g" `grep "cdh" -rl pom.xml`
  $> sed -i "s/cdh5.7/cdh5.16/g" `grep "cdh" -rl pom.xml`
  
  
<profile>
  <id>cdh5.16</id>
  <properties>
    <hadoop2.version>2.6.0-cdh5.16.2</hadoop2.version>
    <yarn.version>2.6.0-cdh5.16.2</yarn.version>
    <hive.version>1.1.0-cdh5.16.2</hive.version>
    <hive-hcatalog.version>1.1.0-cdh5.16.2</hive-hcatalog.version>
    <hbase-hadoop2.version>1.2.0-cdh5.16.2</hbase-hadoop2.version>
    <zookeeper.version>3.4.5-cdh5.16.2</zookeeper.version>
  </properties>
  ......

CDH各个版本下载&&组件版本
https://blog.csdn.net/xiaozhaoshigedasb/article/details/89020421

5.x的CDH各个组件版本信息
https://www.cloudera.com/documentation/enterprise/release-notes/topics/rg_cdh_vd.html

6.x的CDH各个组件版本信息
https://www.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_62_packaging.html

8.5 关闭DocLint特性[可选]

  Java 8 新增了DocLint特性，这个特性主要是在开发阶段生产javadoc文档之前就检查Javadoc注释的错误，并且链接到源代码；如果javadoc的注释有错误，不生产javadoc
 <additionalparam>-Xdoclint:none</additionalparam>

8.6 编译报错问题排查

最终使用的编译命令

build/script/package.sh -Dmaven.test.skip=true -Dcheckstyle.skip -Pcdh5.16.2 -X

编译过程中的报错排查

使用
build/script/package.sh -DskipTests -Pcdh5.16.2 -Papache-release -Dcheckstyle.skip
-X
进行编译一直报错：
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-gpg-plugin:1.6:sign (sign-release-artifacts) on project kylin: Exit code: 2 -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
使用build/script/package.sh -DskipTests -Pcdh5.16这个编译则没有


[ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (process-resource-bundles) on project kylin-engine-spark: Failed to resolve dependencies for one or more projects in the reactor. Reason: No versions are present in the repository for the artifact with a range [1.8,2.0)
[ERROR] commons-codec:commons-codec:jar:null
[ERROR] 
[ERROR] from the specified remote repositories:
[ERROR] central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
[ERROR] conjars (http://conjars.org/repo/, releases=true, snapshots=true),
[ERROR] cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=true),
[ERROR] shibboleth (https://build.shibboleth.net/nexus/content/repositories/releases/, releases=true, snapshots=true),
[ERROR] nexus (http://repository.kyligence.io:8081/repository/maven-public/, releases=true, snapshots=true),
[ERROR] apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
[ERROR] sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
[ERROR] Path to dependency:
[ERROR] 1) org.apache.kylin:kylin-engine-spark:jar:2.6.0
[ERROR] 2) org.apache.spark:spark-core_2.11:jar:2.3.2
[ERROR] 3) net.java.dev.jets3t:jets3t:jar:0.9.4
[ERROR] -> [Help 1]
原因：无法将依赖下载下来，maven仓库中没有找到对应的文件，setting.xml中添加阿里云的maven仓库

******添加阿里云仓库即可******
    <mirror>
            <id>nexus-aliyun</id>
            <mirrorOf>central</mirrorOf>
            <name>Nexus aliyun</name>
            <url>http://maven.aliyun.com/nexus/content/groups/public</url>
    </mirror>

    <mirror>
             <id>CN</id>
             <name>OSChina Central</name>
             <url>http://maven.oschina.net/content/groups/public/</url>
             <mirrorOf>central</mirrorOf>
    </mirror>

    <mirror>
             <id>alimaven</id>
             <mirrorOf>central</mirrorOf>
             <name>aliyun maven</name>
             <url>https://maven.aliyun.com/nexus/content/repositories/central/</url>
    </mirror>

    <mirror>
             <id>jboss-public-repository-group</id>
            <mirrorOf>central</mirrorOf>
            <name>JBoss Public Repository Group</name>
            <url>https://repository.jboss.org/nexus/content/groups/public</url>
    </mirror>


[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile (scala-compile-first) on project kylin-engine-spark: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile failed. CompileFailed -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile (scala-compile-first) on project kylin-engine-spark: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile failed.
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.PluginExecutionException: Execution scala-compile-first of goal net.alchim31.maven:scala-maven-plugin:4.4.0:compile failed.
原因：在pom.xml中找到net.alchim31.maven，并取消注释(在发布时取消注释)，使用的是3.4.1版本
<plugin>
<groupId>net.alchim31.maven</groupId>
<artifactId>scala-maven-plugin</artifactId>
<!-- uncomment this on release -->
<version>3.4.1</version>
</plugin>


修正后继续编译，仍然报错：
Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.4.1:add-source (scala-compile-first) on project kylin-engine-spark: The plugin net.alchim31.maven:scala-maven-plugin:3.4.1 requires Maven version 3.5.3
需要将maven版本改为3.5.3


[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.5.1:testCompile (default-testCompile) on project kylin-source-jdbc: Compilation failure
[ERROR] /opt/source/apache-kylin-2.6.0/source-jdbc/src/test/java/org/apache/kylin/source/jdbc/JdbcExplorerTest.java:[89,26] error: incompatible types: inferred type does not conform to upper bound(s)
[ERROR] 
[ERROR] -> [Help 1]
原因：
使用错误命令`build/script/package.sh -DskipTests -Dcheckstyle.skip -Pcdh5.16`
-Dmaven.test.skip=true  不但跳过单元测试的运行，也跳过测试代码的编译
-DskipTests  跳过单元测试，但是会继续编译
应该使用`build/script/package.sh -Dmaven.test.skip=true -Dcheckstyle.skip -Pcdh5.16`


    [ERROR] Failed to execute goal on project kylin-server-base: Could not resolve dependencies for project org.apache.kylin:kylin-server-base:jar:2.6.0: Failed to collect dependencies at org.springframework.security.extensions:spring-security-saml2-core:jar:1.0.2.RELEASE -> org.opensaml:opensaml:jar:2.6.6: Failed to read artifact descriptor for org.opensaml:opensaml:jar:2.6.6: Could not transfer artifact org.opensaml:opensaml:pom:2.6.6 from/to spring-snapshots (http://repo.spring.io/libs-snapshot): Access denied to: http://repo.spring.io/libs-snapshot/org/opensaml/opensaml/2.6.6/opensaml-2.6.6.pom , ReasonPhrase:Forbidden. -> [Help 1]
    
    [ERROR] Failed to execute goal on project kylin-server-base: Could not resolve dependencies for project org.apache.kylin:kylin-server-base:jar:2.6.0: Failed to collect dependencies at org.springframework.security.extensions:spring-security-saml2-core:jar:1.0.2.RELEASE -> org.opensaml:opensaml:jar:2.6.6: Failed to read artifact descriptor for org.opensaml:opensaml:jar:2.6.6: Could not transfer artifact net.shibboleth:parent-v2:pom:4 from/to spring-snapshots (http://repo.spring.io/libs-snapshot): Access denied to: http://repo.spring.io/libs-snapshot/net/shibboleth/parent-v2/4/parent-v2-4.pom , ReasonPhrase:Forbidden. -> [Help 1]
    
    [ERROR] Failed to execute goal on project kylin-server-base: Could not resolve dependencies for project org.apache.kylin:kylin-server-base:jar:2.6.0: Failed to collect dependencies at org.springframework.security.extensions:spring-security-saml2-core:jar:1.0.2.RELEASE -> org.opensaml:opensaml:jar:2.6.6 -> org.opensaml:openws:jar:1.5.6: Failed to read artifact descriptor for org.opensaml:openws:jar:1.5.6: Could not transfer artifact org.opensaml:openws:pom:1.5.6 from/to spring-snapshots (http://repo.spring.io/libs-snapshot): Access denied to: http://repo.spring.io/libs-snapshot/org/opensaml/openws/1.5.6/openws-1.5.6.pom , ReasonPhrase:Forbidden. -> [Help 1]
    
    [ERROR] Failed to execute goal on project kylin-server-base: Could not resolve dependencies for project org.apache.kylin:kylin-server-base:jar:2.6.0: Failed to collect dependencies at org.springframework.security.extensions:spring-security-saml2-core:jar:1.0.2.RELEASE -> org.opensaml:opensaml:jar:2.6.6 -> org.opensaml:openws:jar:1.5.6 -> org.opensaml:xmltooling:jar:1.4.6: Failed to read artifact descriptor for org.opensaml:xmltooling:jar:1.4.6: Could not transfer artifact org.opensaml:xmltooling:pom:1.4.6 from/to spring-snapshots (http://repo.spring.io/libs-snapshot): Access denied to: http://repo.spring.io/libs-snapshot/org/opensaml/xmltooling/1.4.6/xmltooling-1.4.6.pom , ReasonPhrase:Forbidden. -> [Help 1]
    解决：
    手动下载https://mvnrepository.com/artifact/org.opensaml/opensaml/2.6.6 jar
    手动下载https://build.shibboleth.net/nexus/content/repositories/releases/net/shibboleth/parent-v2/4/parent-v2-4.pom
    手动下载https://mvnrepository.com/artifact/org.opensaml/openws/1.5.6 jar和pom
    手动下载https://mvnrepository.com/artifact/org.opensaml/xmltooling/1.4.6 jar和pom
    
# 找不到spark依赖  添加阿里云仓库即可
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (process-resource-bundles) on project kylin-engine-spark: Failed to resolve dependencies for one or more projects in the reactor. Reason: No versions are present in the repository for the artifact with a range [1.8,2.0)
[ERROR]   commons-codec:commons-codec:jar:null
[ERROR] 
[ERROR] from the specified remote repositories:
[ERROR]   central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
[ERROR]   conjars (http://conjars.org/repo/, releases=true, snapshots=true),
[ERROR]   cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=true),
[ERROR]   shibboleth (https://build.shibboleth.net/nexus/content/repositories/releases/, releases=true, snapshots=true),
[ERROR]   nexus (http://repository.kyligence.io:8081/repository/maven-public/, releases=true, snapshots=true),
[ERROR]   apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
[ERROR]   sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
[ERROR] Path to dependency: 
[ERROR]         1) org.apache.kylin:kylin-engine-spark:jar:2.6.0
[ERROR]         2) org.apache.spark:spark-core_2.11:jar:2.3.2
[ERROR]         3) net.java.dev.jets3t:jets3t:jar:0.9.4
[ERROR] 
[ERROR] 
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-remote-resources-plugin:1.5:process (process-resource-bundles) on project kylin-engine-spark: Failed to resolve dependencies for one or more projects in the reactor. Reason: No versions are present in the repository for the artifact with a range [1.8,2.0)
  commons-codec:commons-codec:jar:null

from the specified remote repositories:
  central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
  conjars (http://conjars.org/repo/, releases=true, snapshots=true),
  cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=true),
  shibboleth (https://build.shibboleth.net/nexus/content/repositories/releases/, releases=true, snapshots=true),
  nexus (http://repository.kyligence.io:8081/repository/maven-public/, releases=true, snapshots=true),
  apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
  sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
Path to dependency: 
        1) org.apache.kylin:kylin-engine-spark:jar:2.6.0
        2) org.apache.spark:spark-core_2.11:jar:2.3.2
        3) net.java.dev.jets3t:jets3t:jar:0.9.4


    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: Failed to resolve dependencies for one or more projects in the reactor. Reason: No versions are present in the repository for the artifact with a range [1.8,2.0)
  commons-codec:commons-codec:jar:null

from the specified remote repositories:
  central (http://repo.maven.apache.org/maven2, releases=true, snapshots=false),
  conjars (http://conjars.org/repo/, releases=true, snapshots=true),
  cloudera (https://repository.cloudera.com/artifactory/cloudera-repos/, releases=true, snapshots=true),
  shibboleth (https://build.shibboleth.net/nexus/content/repositories/releases/, releases=true, snapshots=true),
  nexus (http://repository.kyligence.io:8081/repository/maven-public/, releases=true, snapshots=true),
  apache.snapshots (https://repository.apache.org/snapshots, releases=false, snapshots=true),
  sonatype-nexus-snapshots (https://oss.sonatype.org/content/repositories/snapshots, releases=false, snapshots=true)
Path to dependency: 
        1) org.apache.kylin:kylin-engine-spark:jar:2.6.0
        2) org.apache.spark:spark-core_2.11:jar:2.3.2
        3) net.java.dev.jets3t:jets3t:jar:0.9.4

[WARNING] Could not transfer metadata commons-codec:commons-codec/maven-metadata.xml from/to central (http://repo.maven.apache.org/maven2): Transfer failed for http://repo.maven.apache.org/maven2/commons-codec/commons-codec/maven-metadata.xml 501 HTTPS Required
org.eclipse.aether.transfer.MetadataTransferException: Could not transfer metadata commons-codec:commons-codec/maven-metadata.xml from/to central (http://repo.maven.apache.org/maven2): Transfer failed for http://repo.maven.apache.org/maven2/commons-codec/commons-codec/maven-metadata.xml 501 HTTPS Required
    at org.eclipse.aether.connector.basic.MetadataTransportListener.transferFailed (MetadataTransportListener.java:52)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:369)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)
Caused by: org.apache.maven.wagon.TransferFailedException: Transfer failed for http://repo.maven.apache.org/maven2/commons-codec/commons-codec/maven-metadata.xml 501 HTTPS Required
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1196)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
    at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
    at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
    at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
    at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunner.run (WagonTransporter.java:567)
    at org.eclipse.aether.transport.wagon.WagonTransporter.execute (WagonTransporter.java:435)
    at org.eclipse.aether.transport.wagon.WagonTransporter.get (WagonTransporter.java:412)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:457)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:364)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)
[WARNING] Could not transfer metadata commons-codec:commons-codec/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/maven-metadata.xml 401 Unauthorized
org.eclipse.aether.transfer.MetadataTransferException: Could not transfer metadata commons-codec:commons-codec/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/maven-metadata.xml 401 Unauthorized
    at org.eclipse.aether.connector.basic.MetadataTransportListener.transferFailed (MetadataTransportListener.java:52)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:369)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)
Caused by: org.apache.maven.wagon.authorization.AuthorizationException: Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/maven-metadata.xml 401 Unauthorized
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1180)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
    at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
    at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
    at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
    at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunner.run (WagonTransporter.java:567)
    at org.eclipse.aether.transport.wagon.WagonTransporter.execute (WagonTransporter.java:435)
    at org.eclipse.aether.transport.wagon.WagonTransporter.get (WagonTransporter.java:412)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:457)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:364)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:624)
    at java.lang.Thread.run (Thread.java:748)
[DEBUG] Could not find metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in local (/root/.m2/repository)
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in http://conjars.org/repo/ was cached in the local repository, resolution will not be reattempted until the update interval of conjars has elapsed or updates are forced
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://repository.cloudera.com/artifactory/cloudera-repos/ was cached in the local repository, resolution will not be reattempted until the update interval of cloudera has elapsed or updates are forced
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://build.shibboleth.net/nexus/content/repositories/releases/ was cached in the local repository, resolution will not be reattempted until the update interval of shibboleth has elapsed or updates are forced
[DEBUG] Skipped remote request for commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml, locally cached metadata up-to-date.
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://oss.sonatype.org/content/repositories/snapshots was cached in the local repository, resolution will not be reattempted until the update interval of sonatype-nexus-snapshots has elapsed or updates are forced
[DEBUG] Using transporter WagonTransporter with priority -1.0 for http://repository.kyligence.io:8081/repository/maven-public/
[DEBUG] Using connector BasicRepositoryConnector with priority 0.0 for http://repository.kyligence.io:8081/repository/maven-public/
Downloading from nexus: http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml
[DEBUG] Writing tracking file /root/.m2/repository/commons-codec/commons-codec/1.15-SNAPSHOT/resolver-status.properties
[WARNING] Could not transfer metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
org.eclipse.aether.transfer.MetadataTransferException: Could not transfer metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
    at org.eclipse.aether.connector.basic.MetadataTransportListener.transferFailed (MetadataTransportListener.java:52)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:369)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver$1.execute (DefaultMetadataResolver.java:494)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolve (DefaultMetadataResolver.java:350)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolveMetadata (DefaultMetadataResolver.java:181)
    at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion (DefaultVersionResolver.java:213)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:204)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:171)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.resolveCachedArtifactDescriptor (DefaultDependencyCollector.java:541)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.getArtifactDescriptorResult (DefaultDependencyCollector.java:524)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:412)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies (DefaultDependencyCollector.java:254)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies (DefaultRepositorySystem.java:284)
    at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:169)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:243)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:147)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:248)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:202)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.wagon.authorization.AuthorizationException: Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1180)
    at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:1138)
    at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126)
    at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88)
    at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61)
    at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunner.run (WagonTransporter.java:567)
    at org.eclipse.aether.transport.wagon.WagonTransporter.execute (WagonTransporter.java:435)
    at org.eclipse.aether.transport.wagon.WagonTransporter.get (WagonTransporter.java:412)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:457)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:364)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:644)
    at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:235)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver$ResolveTask.run (DefaultMetadataResolver.java:573)
    at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver$1.execute (DefaultMetadataResolver.java:494)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolve (DefaultMetadataResolver.java:350)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolveMetadata (DefaultMetadataResolver.java:181)
    at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion (DefaultVersionResolver.java:213)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:204)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:171)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.resolveCachedArtifactDescriptor (DefaultDependencyCollector.java:541)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.getArtifactDescriptorResult (DefaultDependencyCollector.java:524)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:412)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies (DefaultDependencyCollector.java:254)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies (DefaultRepositorySystem.java:284)
    at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:169)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:243)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:147)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:248)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:202)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[DEBUG] Could not find metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in local (/root/.m2/repository)
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in http://conjars.org/repo/ was cached in the local repository, resolution will not be reattempted until the update interval of conjars has elapsed or updates are forced
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://repository.cloudera.com/artifactory/cloudera-repos/ was cached in the local repository, resolution will not be reattempted until the update interval of cloudera has elapsed or updates are forced
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://build.shibboleth.net/nexus/content/repositories/releases/ was cached in the local repository, resolution will not be reattempted until the update interval of shibboleth has elapsed or updates are forced
[DEBUG] Skipped remote request for commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml, already updated during this session.
[WARNING] Failure to transfer commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from http://repository.kyligence.io:8081/repository/maven-public/ was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced. Original error: Could not transfer metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
org.eclipse.aether.transfer.MetadataTransferException: Failure to transfer commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from http://repository.kyligence.io:8081/repository/maven-public/ was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced. Original error: Could not transfer metadata commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml from/to nexus (http://repository.kyligence.io:8081/repository/maven-public/): Authentication failed for http://repository.kyligence.io:8081/repository/maven-public/commons-codec/commons-codec/1.15-SNAPSHOT/maven-metadata.xml 401 Unauthorized
    at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.newException (DefaultUpdateCheckManager.java:336)
    at org.eclipse.aether.internal.impl.DefaultUpdateCheckManager.checkMetadata (DefaultUpdateCheckManager.java:296)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolve (DefaultMetadataResolver.java:302)
    at org.eclipse.aether.internal.impl.DefaultMetadataResolver.resolveMetadata (DefaultMetadataResolver.java:181)
    at org.apache.maven.repository.internal.DefaultVersionResolver.resolveVersion (DefaultVersionResolver.java:213)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.loadPom (DefaultArtifactDescriptorReader.java:211)
    at org.apache.maven.repository.internal.DefaultArtifactDescriptorReader.readArtifactDescriptor (DefaultArtifactDescriptorReader.java:171)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.resolveCachedArtifactDescriptor (DefaultDependencyCollector.java:541)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.getArtifactDescriptorResult (DefaultDependencyCollector.java:524)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:412)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.doRecurse (DefaultDependencyCollector.java:509)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:461)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.processDependency (DefaultDependencyCollector.java:365)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.process (DefaultDependencyCollector.java:352)
    at org.eclipse.aether.internal.impl.collect.DefaultDependencyCollector.collectDependencies (DefaultDependencyCollector.java:254)
    at org.eclipse.aether.internal.impl.DefaultRepositorySystem.collectDependencies (DefaultRepositorySystem.java:284)
    at org.apache.maven.project.DefaultProjectDependenciesResolver.resolve (DefaultProjectDependenciesResolver.java:169)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.getDependencies (LifecycleDependencyResolver.java:243)
    at org.apache.maven.lifecycle.internal.LifecycleDependencyResolver.resolveProjectDependencies (LifecycleDependencyResolver.java:147)
    at org.apache.maven.lifecycle.internal.MojoExecutor.ensureDependenciesAreResolved (MojoExecutor.java:248)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:202)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[DEBUG] Skipped remote request for commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml, locally cached metadata up-to-date.
[DEBUG] Failure to find commons-codec:commons-codec:1.15-SNAPSHOT/maven-metadata.xml in https://oss.sonatype.org/content/repositories/snapshots was cached in the local repository, resolution will not be reattempted until the update interval of sonatype-nexus-snapshots has elapsed or updates are forced

# bower与npm错误
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.1.2 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN base@0.0.1 No license field.
npm ERR! Linux 3.10.0-514.26.2.el7.x86_64
npm ERR! argv "/usr/local/node/node-v6.17.1-linux-x64/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v6.17.1
npm ERR! npm  v3.10.10
npm ERR! path /kylin/source/apache-kylin-2.6.0/webapp/node_modules/.staging/@types/color-name-28c4216b
npm ERR! code ENOENT
npm ERR! errno -2
npm ERR! syscall rename

npm ERR! enoent ENOENT: no such file or directory, rename '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/.staging/@types/color-name-28c4216b' -> '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/tap/node_modules/@types/color-name'
npm ERR! enoent ENOENT: no such file or directory, rename '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/.staging/@types/color-name-28c4216b' -> '/kylin/source/apache-kylin-2.6.0/webapp/node_modules/tap/node_modules/@types/color-name'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent 

npm ERR! Please include the following file with any support request:
npm ERR!     /kylin/source/apache-kylin-2.6.0/webapp/npm-debug.log

8.7 最终编译成功

[INFO]
[INFO] --- maven-install-plugin:2.5.2:install (default-install) @ kylin-tomcat-ext ---
[INFO] Installing /opt/source/apache-kylin-2.6.0/tomcat-ext/target/kylin-tomcat-ext-2.6.0.jar to /home/hadoop/.m2/repository/org/
apache/kylin/kylin-tomcat-ext/2.6.0/kylin-tomcat-ext-2.6.0.jar
[INFO] Installing /opt/source/apache-kylin-2.6.0/tomcat-ext/pom.xml to /home/hadoop/.m2/repository/org/apache/kylin/kylin-tomcatext/2.6.0/kylin-tomcat-ext-2.6.0.pom
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Kylin 2.6.0 ................................. SUCCESS [ 6.896 s]
[INFO] Apache Kylin - Core Common ......................... SUCCESS [ 9.497 s]
[INFO] Apache Kylin - Core Metadata ....................... SUCCESS [ 11.527 s]
[INFO] Apache Kylin - Core Dictionary ..................... SUCCESS [ 7.266 s]
[INFO] Apache Kylin - Core Cube ........................... SUCCESS [ 9.569 s]
[INFO] Apache Kylin - Core Metrics ........................ SUCCESS [ 4.294 s]
[INFO] Apache Kylin - Core Job ............................ SUCCESS [ 6.162 s]
[INFO] Apache Kylin - Core Storage ........................ SUCCESS [ 4.810 s]
[INFO] Apache Kylin - MapReduce Engine .................... SUCCESS [ 11.646 s]
[INFO] Apache Kylin - Spark Engine ........................ SUCCESS [ 36.087 s]
[INFO] Apache Kylin - Hive Source ......................... SUCCESS [ 16.175 s]
[INFO] Apache Kylin - DataSource SDK ...................... SUCCESS [ 9.821 s]
[INFO] Apache Kylin - Jdbc Source ......................... SUCCESS [ 6.607 s]
[INFO] Apache Kylin - Kafka Source ........................ SUCCESS [ 7.157 s]
[INFO] Apache Kylin - Cache ............................... SUCCESS [ 5.200 s]
[INFO] Apache Kylin - HBase Storage ....................... SUCCESS [ 21.739 s]
[INFO] Apache Kylin - Query ............................... SUCCESS [ 10.917 s]
[INFO] Apache Kylin - Metrics Reporter Hive ............... SUCCESS [ 8.855 s]
[INFO] Apache Kylin - Metrics Reporter Kafka .............. SUCCESS [ 5.503 s]
[INFO] Apache Kylin - REST Server Base .................... SUCCESS [01:21 min]
[INFO] Apache Kylin - REST Server ......................... SUCCESS [01:47 min]
[INFO] Apache Kylin - JDBC Driver ......................... SUCCESS [01:29 min]
[INFO] Apache Kylin - Assembly ............................ SUCCESS [02:21 min]
[INFO] Apache Kylin - Tool ................................ SUCCESS [ 33.614 s]
[INFO] Apache Kylin - Tool Assembly ....................... SUCCESS [ 22.368 s]
[INFO] Apache Kylin - Integration Test .................... SUCCESS [01:08 min]
[INFO] Apache Kylin - Tomcat Extension 2.6.0 .............. SUCCESS [ 6.227 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12:31 min
[INFO] Finished at: 2020-05-17T04:29:36+08:00
[INFO] ------------------------------------------------------------------------
....
apache-kylin-2.6.0-bin/bin/check-port-availability.sh
apache-kylin-2.6.0-bin/bin/build-incremental-cube.sh
apache-kylin-2.6.0-bin/bin/find-spark-dependency.sh
apache-kylin-2.6.0-bin/bin/check-migration-acl.sh
apache-kylin-2.6.0-bin/bin/find-hadoop-conf-dir.sh
apache-kylin-2.6.0-bin/bin/find-hbase-dependency.sh
apache-kylin-2.6.0-bin/bin/check-env.sh
apache-kylin-2.6.0-bin/bin/set-java-home.sh
apache-kylin-2.6.0-bin/bin/sample.sh
apache-kylin-2.6.0-bin/bin/find-kafka-dependency.sh
apache-kylin-2.6.0-bin/bin/metastore.sh
apache-kylin-2.6.0-bin/bin/sample-streaming.sh
apache-kylin-2.6.0-bin/bin/diag.sh
apache-kylin-2.6.0-bin/bin/find-hive-dependency.sh
apache-kylin-2.6.0-bin/bin/kylin.sh
apache-kylin-2.6.0-bin/bin/check-hive-usability.sh
apache-kylin-2.6.0-bin/bin/system-cube.sh
Package ready: dist/apache-kylin-2.6.0-bin.tar.gz
真正意义上是编译成功了 基于cdh5.16.2

# 最终编译包在根目录 dist包下
/kylin/source/apache-kylin-2.6.0/dist/apache-kylin-2.6.0-bin.tar.gz
[root@hadoop003 dist]# pwd
/kylin/source/apache-kylin-2.6.0/dist
[root@hadoop003 dist]# ls -lah
total 337M
drwxr-xr-x  2 root root 4.0K Jun 28 15:22 .
drwxrwxrwx 36 root root 4.0K Jun 28 15:33 ..
-rw-r--r--  1 root root 337M Jun 28 15:37 apache-kylin-2.6.0-bin.tar.gz

9.运行kylin所需要环境

9.1 Hadoop部署

目录：/opt/app/hadoop-2.6.0-cdh5.16.2/etc/hadoop
corr-site.xml
<configuration>
	<property>
	<name>fs.defaultFS</name>
	<value>hdfs://hadoop001:8020</value>
	</property>

	<property>
	<name>hadoop.tmp.dir</name>
	<value>/opt/app/tmp</value>
	</property>
</configuration>

/opt/app/tmp目录下的文件信息：
[hadoop@hadoop001 hadoop]$ cd /opt/app/tmp
[hadoop@hadoop001 tmp]$ ll
total 8
drwxrwxr-x. 5 hadoop hadoop 4096 Nov  4  2017 dfs
drwxr-xr-x. 5 hadoop hadoop 4096 May 20 08:27 nm-local-dir

hdfs-site.xml
<configuration>
	<property>
	<name>dfs.replication</name>
	<value>1</value>
	</property>
</configuration>

mapred-site.xml
<configuration>
	<property>
	<name>mapreduce.framework.name</name>
	<value>yarn</value>
	</property>

	<!--如果不配置jobhistory，那么在kylin cube构建的时候会产生报错-->
	<property>
	<name>mapreduce.jobhistory.address</name>
	<value>hadoop001:10020</value>
	</property>
</configuration>

yarn-site.xml
<configuration>
	<property>
	<name>yarn.nodemanager.aux-services</name>
	<value>mapreduce_shuffle</value>
	</property>
</configuration>

slaves 
localhost

9.2 Hive部署

一般会用于大多数公司的离线数据仓库的构建，SQL on Hadoop的框架，我们可以通过写hive sql的方式去运行一个job，这个job会对应的转换为mr作业进行运行
[拓展]hive on mr/spark/tez

hive-env.sh
HADOOP_HOME=/opt/app/hadoop-2.6.0-cdh5.16.2

hive-site.xml
<configuration> 
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/ruozedata_biproject?createDatabaseIfNotExist=true</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
</property>
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>root</value>
</property>
</configuration>
为什么需要配置mysql相关的内容?
hive的数据组成：data(hdfs) + meta data(mysql)

启动hive报错：
hive> show databases;
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
排查方向：去看具体的log  /tmp/hadoop/hive.log
报错日志：
Caused by: org.datanucleus.store.rdbms.connectionpool.DatastoreDriverNotFoundException: The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
        at org.datanucleus.store.rdbms.connectionpool.AbstractConnectionPoolFactory.loadDriver(AbstractConnectionPoolFactory.java:58)
        at org.datanucleus.store.rdbms.connectionpool.BoneCPConnectionPoolFactory.createConnectionPool(BoneCPConnectionPoolFactory.java:54)
        at org.datanucleus.store.rdbms.ConnectionFactoryImpl.generateDataSources(ConnectionFactoryImpl.java:238)
        ... 81 more
原因：缺少jar包 mysql-connector-java-5.1.39-bin.jar 拷贝到$HIVE_HOME/lib下即可

修改过后启动hive仍然报错：
hive> show databases;
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
排查方向：去看具体的log  /tmp/hadoop/hive.log
报错日志：
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:589)
        at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:211)
        at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:300)
        ... 89 more
原因：启动mysql

最终运行成功：
hive> show databases;
OK
default
kylin
ruozedata
test

9.3 HBase部署

hbase-env.sh
export JAVA_HOME=/opt/app/jdk1.8.0_45
如果不配置的话，启动会报错
[hadoop@hadoop001 conf]$ start-hbase.sh 
192.168.26.131: +======================================================================+
192.168.26.131: |                    Error: JAVA_HOME is not set                       |
192.168.26.131: +----------------------------------------------------------------------+
192.168.26.131: | Please download the latest Sun JDK from the Sun Java web site        |
192.168.26.131: |     > http://www.oracle.com/technetwork/java/javase/downloads        |
192.168.26.131: |                                                                      |
192.168.26.131: | HBase requires Java 1.7 or later.                                    |
192.168.26.131: +======================================================================+
starting master, logging to /opt/app/hbase-1.2.0-cdh5.16.2/logs/hbase-hadoop-master-hadoop001.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
starting regionserver, logging to /opt/app/hbase-1.2.0-cdh5.16.2/logs/hbase-hadoop-1-regionserver-hadoop001.out

hbase-site.xml
<configuration>
	<property>
	<name>hbase.rootdir</name>
	<value>hdfs://192.168.26.131:8020/hbase</value>
	</property>

	<property>
	<name>hbase.cluster.distributed</name>
	<value>true</value>
	</property>

	<property>
	<name>hbase.zookeeper.quorum</name>
	<value>192.168.26.131</value>
	</property>
</configuration>

对应的进程：HMaster、HRegionServer、HQuorumPeer
web ui：http://hadoop001:60010/master-status

9.4 Kylin部署

需要去编译源码  基于kylin的2.6.0版本 cdh5.16.2

kylin.properties
kylin.env.hadoop-conf-dir=/opt/app/hadoop-2.6.0-cdh5.16.2/etc/hadoop

启动kylin：
[hadoop@hadoop001 bin]$ ./kylin.sh start

看到如下信息输出，即启动成功：
A new Kylin instance is started by hadoop. To stop it, run 'kylin.sh stop'
Check the log at /opt/app/apache-kylin-2.6.0/logs/kylin.log
Web UI is at http://<hostname>:7070/kylin
You have new mail in /var/spool/mail/hadoop

[hadoop@hadoop001 bin]$ ps -ef | grep kylin

web ui：http://hadoop001:7070/kylin/login   ADMIN/KYLIN