Appche Kylin集群部署以及汇总
官方下载文档https://archive.apache.org/dist/kylin/
kylin部署环境支持
• Hadoop: 2.7+, 3.1+ (since v2.5)
• Hive: 0.13 - 1.2.1+
• HBase: 1.1+, 2.0 (since v2.5)
• Spark (可选) 2.3.0+
• Kafka (可选) 1.0.0+ (since v2.5)
• JDK: 1.8+ (since v2.5)
• OS: Linux only, CentOS 6.5+ or Ubuntu 16.0.4+
配置环境变量
KYLIN_HOME=/usr/hdp/3.0.0.0-1634/kylin
启动时可用kylin/bin/check-env.sh检查集群环境是否可用,如不可用,会有相应的报错信息
启动报错汇总
Something wrong with Hive CLI or Beeline, please execute Hive CLI or Beeline CLI in terminal to find the root cause
解决方法:
vim bin/find-hive-dependency.sh
hive_env=hive ${hive_conf_properties} -e set 2>&1 | grep 'env:CLASSPATH'
中的变量 ${hive_conf_properties}去掉(未配置此变量),即修改为
hive_env=hive -e set 2>&1 | grep 'env:CLASSPATH'
${KYLIN_HOME}/tomcat/conf/.keystore
解决方法:在kylin内置tomcat的server.xml中里边有个对https的支持那一段没启用的话 注释掉
<Connector port="7443" protocol="org.apache.coyote.http11.Http11Protocol"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
keystoreFile="conf/.keystore" keystorePass="changeit"
clientAuth="false" sslProtocol="TLS" />
Caused by: java.lang.NoClassDefFoundError: org/apache/commons/configuration/ConfigurationException
解决方法:下载commons-configuration-*.jar 复制到kylin的tomcat/lib 下
Error: Could not open client transport for any of the Server URI's in ZooKeeper: Failed to open new session: java.lang.IllegalArgumentException: Cannot modify dfs.replication at runtime. It is not in list of params that are allowed to be modified at runtime (state=08S01,code=0)
解决方法:kylin/conf/kylin_hive_conf.xml中注释掉dfs.replication这个属性,还有mapreduce.job.split.metainfo.maxsize这个属性也注释掉
LF4J: Class path contains multiple SLF4Jbindings.
Error: Error while processing statement: Cannot modify mapred.job.name at runtime. It is not in list of params that are allowed to be modified at runtime (state=42000,code=1)
解决方法:hive-site.xml增加自定义参数
hive.security.authorization.sqlstd.confwhitelist
mapred.|hive.|mapreduce.|spark.
hive.security.authorization.sqlstd.confwhitelist.append
mapred.|hive.|mapreduce.|spark.