一、安装整合
phoenix没有与CDH高版本(cdh6.x)的整合,这里phoenix为apache原生版本
组件:CDH6.0、Hbase2.0、Phoenix5.0
相关文档:
https://blog.csdn.net/zimiao552147572/article/details/87286352
二、Spark+Phoenix连接读写测试
1、本地环境
依赖:
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-server</artifactId>
<version>2.0.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.hbase/hbase-client -->
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>2.0.0</version>
</dependency>
<dependency>
<groupId>org.apache.phoenix</groupId>
<artifactId>phoenix-core</artifactId>
<version>5.0.0-HBase-2.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.phoenix/phoenix-spark -->
<dependency>
<groupId>org.apache.phoenix</groupId>
<artifactId>phoenix-spark</artifactId>
<version>5.0.0-HBase-2.0</version>
</dependency>
拷贝hbase-site.xml到resource目录
2、集群环境
问题:各种NoClassFound
拷贝如下相关jar包,过程比较繁琐,需要根据报错信息去找依赖
disruptor-3.3.6.jar
hbase-annotations-2.0.0.jar
hbase-client-2.0.0.jar
hbase-common-2.0.0.jar
hbase-hadoop2-compat-2.0.0.jar
hbase-hadoop-compat-2.0.0.jar
hbase-mapreduce-2.0.0.jar
hbase-procedure-2.0.0.jar
hbase-protocol-2.0.0.jar
hbase-protocol-shaded-2.0.0.jar
hbase-replication-2.0.0.jar
hbase-server-2.0.0.jar
hbase-shaded-miscellaneous-2.1.0.jar
hbase-shaded-netty-2.1.0.jar
hbase-shaded-protobuf-2.1.0.jar
phoenix-spark-5.0.0-HBase-2.0.jar
phoenix-core-5.0.0-HBase-2.0.jar
tephra-core-0.14.0-incubating.jar
twill-api-0.8.0.jar
twill-core-0.8.0.jar
twill-discovery-api-0.8.0.jar
twill-zookeeper-0.8.0.jar
使用 spark-submit提交时需要指定客户端环境的hbase-site.xml位置
--files /etc/hbase/conf/hbase-site.xml
三、使用jdbc连接phoenix并查询测试
使用squirrel工具,最终有点问题
最后换用DBeaver有效。相关链接: https://blog.csdn.net/darkbreak_up/article/details/102861213