大数据
文章平均质量分 61
飞鱼丶灬
这个作者很懒,什么都没留下…
展开
-
spark 相关
spark原创 2022-06-02 16:28:48 · 233 阅读 · 0 评论 -
HBase + Key-Value Store Indexer + Solr
准备hbase 启动并开启复制配置Solr 启动Key-Value Store Indexer 启动Lily HBase Indexer 的 Java 堆栈大小(字节)调整1.solr 创建collectionMIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX2.solr 添加字段3.solr 设置自动提交时间发送POST请求curl http://n1:8983/solr/MIDDLE_PLATFORM_MP_HBASE_SOLR_INDEX/config原创 2020-08-21 15:35:21 · 305 阅读 · 0 评论 -
hive sql查询
/*** * 分析引擎查询 * @param dbSourcePageQueryReqDTO * @return * @throws SQLException * @throws ClassNotFoundException */ public List<Map<String, Object>> pageQuery(DbSourcePageQueryReqDTO dbSourcePageQueryReqDTO) th原创 2020-12-28 17:37:16 · 488 阅读 · 0 评论 -
hive
HiveServer2:一、Hive2 Beeline:-Beeline要与HiveServer2配合使用,支持嵌入和远程模式-启动HiveServer2,./bin/hiveserver2 或者 hive --service hiveserver2 &-启动Beeline: 1. ./bin/beeline 或者 hive --service beeline beelin...原创 2019-11-13 10:49:45 · 74 阅读 · 0 评论 -
HBASE 2.0.6 源码编译
忘记安装gcc./setup-x86_64.exe -q -P wget -P gcc-g++ -P make -P diffutils -P libmpfr-devel -P libgmp-devel -P libmpc-devel原创 2021-12-03 10:47:55 · 1760 阅读 · 0 评论 -
HBASE FILTER 源码图
原创 2020-05-19 17:19:46 · 115 阅读 · 0 评论 -
hbase SingleColumnValueFilter 列不存在 无法过滤
当某一行没有要过滤的字段时,SingleColumnValueFilter是默认这一行符合过滤条件的。接下来就要让SingleColumnValueFilter在判断的时候把这个策略改改。查看源码发现是有方法可以更改这个策略的/** * This filter is used to filter cells based on value. It takes a {@link CompareFilter.CompareOp} * operator (equal, greater, not equal原创 2021-03-05 17:28:48 · 651 阅读 · 0 评论 -
hbase 常用 shell
#进命令行hbase shell#批量禁用表disable_all 'MIDDLE_PLATFORM.*'#批量删除表drop_all 'MIDDLE_PLATFORM.*'原创 2019-10-30 17:21:03 · 126 阅读 · 0 评论 -
HDFS HA连接和常用API操作
public class HDFSTest { FileSystem fileSystem; @Before public void setUp() throws IOException { Configuration configuration = new Configuration(); configuration.set("fs.defaultFS", "hdfs://ns1"); configuration.set("dfs.原创 2020-06-29 19:07:20 · 482 阅读 · 0 评论 -
HDFSManager
package com.xinyan.mp.base.manager;import lombok.extern.slf4j.Slf4j;import org.apache.commons.io.IOUtils;import org.apache.hadoop.conf.Configuration;import org.apache.hadoop.fs.*;import org.apache.hadoop.security.UserGroupInformation;import org.spri原创 2020-06-24 11:22:05 · 139 阅读 · 0 评论 -
Impala相关
CREATE TABLE `default`.customers ( id INT, name STRING(32767), email_preferences STRUCT< EMAIL_FORMAT:STRING, FREQUENCY:STRING, CATEGORIES:STRUCT< PROMOS:BOOLEAN, SURVEYS:BOOLEAN >>(32767), addresses MAP<STRING,STRUCT<原创 2020-09-16 10:24:01 · 102 阅读 · 0 评论 -
Impala 常用
impala与LDAP集成后的登录impala-shell -i 127.0.0.1:21000 -u root -l --auth_creds_ok_in_clear[root@AAA ~]$ impala-shell -i 127.0.0.1:21000 -u root -l --auth_creds_ok_in_clearStarting Impala Shell using LDAP-based authenticationLDAP password for root : Connect原创 2021-05-26 17:49:47 · 356 阅读 · 0 评论 -
CDH 5.13.1安装 apache-phoenix-4.14.0-cdh5.13.2
CDH 5.13.1安装 apache-phoenix-4.14.0-cdh5.13.2添加phoenix parcelhttps://archive.apache.org/dist/phoenix/apache-phoenix-4.14.0-cdh5.13.2/parcels/下载分配 激活HBase 配置 phoenix 开启命名空间和二级索引hbase-site....原创 2019-07-30 18:50:52 · 580 阅读 · 0 评论 -
cdh 本地 parcel 包安装 phoenix
ps -ef|grep httpdyum install httpd开机启动systemctl enable httpd.service启动httpd服务systemctl start httpd.servicecd /var/www/htmlmkdir -p phoenix/apache-phoenix-5.1.0-cdh6.0.1/parcels/cd /var/ww...原创 2019-08-21 16:30:48 · 768 阅读 · 0 评论 -
Centos7 bash: jps: 未找到命令...
centos自带open jdk发现jps不能用,此时需要安装另外一个包才能使用通过以下命令进行安装:sudo yum install java-1.8.0-openjdk-devel.x86_64原创 2020-08-07 17:00:39 · 771 阅读 · 0 评论 -
在线安装CDH6.X
##免密#生成ssh-keygen -t rsa#本机免密ssh-copy-id localhost#CM yum源cd /etc/yum.repos.d/wget https://archive.cloudera.com/cm6/6.0.1/redhat7/yum/cloudera-manager.repo vi cloudera-manager.repo#去掉enab...原创 2019-08-01 09:48:39 · 579 阅读 · 0 评论 -
CentOS7安装cdh6.0.1
1原创 2020-08-07 11:12:15 · 350 阅读 · 0 评论 -
Centos7.9部署CDH6.3.2集群(离线版本)
https://docs.cloudera.com/documentation/enterprise/6/6.3/topics/installation.html一.CDH介绍二.安装环境1.系统说明2.版本说明CM 6.3.1,CDH 6.3.2 (6.3.3版本及以上的要收费了)3.组件版本4.安装包准备(1)Cloudera Manger下载cloudera-manager-server-6.3.1-1466458.el7.x86_64cloudera-manager-agent-原创 2022-01-10 10:47:31 · 1670 阅读 · 1 评论 -
Std Err: /usr/sbin/hst: line 460: install-activity-analyzer.sh: command not found
卸载后重新安装AMBARI 2.7.3 报错:While trying to install HDP with Ambari, I often end up getting this error.I have tried multiple times and even downloaded tar.gz file and extracted it as per documentation a...原创 2019-04-18 08:34:07 · 2543 阅读 · 0 评论 -
Apache atlas 安装
Apache atlas 安装1./etc/profile,需要配JAVA_HOME。2.环境:MAVEN3.6/JDK83.下载 wget https://www.apache.org/dyn/closer.cgi/atlas/1.1.0/apache-atlas-1.1.0-sources.tar.gz4.解压 tar xvfz apache-atlas-1.1.0-source...原创 2019-04-15 11:26:35 · 817 阅读 · 0 评论