邢宁

一、安装准备1、下载包wget http://archive.cloudera.com/cloudera-labs/ycsb/parcels/latest/CLABS_YCSB-0.10.0-1.clabs_ycsb1.6.0.p0.1-el7.parcelwget http://archive.cloudera.com/cloudera-labs/ycsb/parcels/latest/CLABS_YCSB-0.10.0-1.clabs_ycsb1.6.0.p0.1-el7.parcel.sha1

一、安装准备1、下载csd包wget http://archive.cloudera.com/spark2/csd/SPARK2_ON_YARN-2.3.0.cloudera2.jar2、下载parcel包wget http://archive.cloudera.com/spark2/parcels/2.3.0.cloudera2/SPARK2-2.3.0.cloudera2-1.cdh5.13.3.p0.316101-el7.parcelwget http://archive.cloudera

一、安装准备1、下载csd包wget http://archive.cloudera.com/csds/kafka/KAFKA-1.2.0.jar2、下载parcel包wget http://archive.cloudera.com/kafka/parcels/2.1.1/KAFKA-2.1.1-1.2.1.1.p0.18-el7.parcelwget http://archive.cloudera.com/kafka/parcels/2.1.1/KAFKA-2.1.1-1.2.1.1.p0.1

环境说明Cloudera Manager：5.13.3CDH: 5.13.3CentOS 7.3 x64JDK 1.8.0_152MySQL 5.7.20关闭防火墙和SELinux (所有节点)配置时间同步（所有节点）#下载CM安装包(主节点)wget http://archive.cloudera.com/cm5/cm/5/cloudera-manager-centos7-cm5.13.3_x86_64.tar.gzwget http://archive.cloudera.com/

1、报错#Centos 7安装 ambari通过Ambari部署系统时，ambari-agent注册失败，在/var/log/ambari-agent的log中可以看到日志显示：ERROR 2018-03-08 15:02:25,223 NetUtil.py:88 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)ERROR 2018-03-08 15:02:25,223 NetUtil.py:89 - S

一、环境准备1、主机列表#vim /etc/hosts192.168.137.128 node1192.168.137.129 node2192.168.137.130 node32、安装epel仓库rpm -ivh epel-release-latest-7.noarch.rpm3、修改主机名, 切记主机名不能出现下划线#修改主机名(/etc/hostname)hostnamectl set-hostname node1hostnamectl set-hostname nod

1、调整默认block大小<property><name>dfs.blocksize</name><value>30m</value></property>2、内存调优vim hadoop-env.shexport HADOOP_NAMENODE_OPTS="-Xmx512m -Xms256m -Dhadoop.security.logger=${HADOOP_SECURITY_LOGGER:-INFO,RFAS} -D

1、报错信息rm: org.apache.hadoop.hdfs.server.namenode.SafeModeException: Cannot delete /hbase. Name node is in safe mode解决办法#hdfs异常重启导致hdfs进入安全模式，退出即可hadoop dfsadmin -safemode leave

1、报错信息2014-02-24 12:15:48,507 WARN [Thread-2] util.DynamicClassLoader (DynamicClassLoader.java:<init>(106)) - Failed to identify the fs of dir hdfs://fulonghadoop/hbase/lib, ignoredjava.io.IOException: No FileSystem for scheme: hdfs解决办法在配置文件中加入

1、解压文件到指定目录tar zxf sqoop-1.4.7.bin__hadoop-2.6.0.tar.gz -C /data2、修改配置文件（1）进入/sqoop/conf目录mv sqoop-env-template.sh sqoop-env.shvim sqoop-env.shexport HADOOP_COMMON_HOME=/data/hadoop/hadoop-2.7.2export HADOOP_MAPRED_HOME=/data/hadoop/hadoop-2.7.2e

1、解压文件到指定目录tar zxf scala-2.11.8.tgz -C /data/tar zxf spark-2.1.1-bin-hadoop2.7.tgz -C /data/2、修改配置文件cd /data/spark-2.1.1-bin-hadoop2.7/confcp spark-env.sh.template spark-env.shcp slaves.template slavesvim spark-env.sh##ssh端口不是22时#export SPARK_S

一、安装1、下载安装包wget http://gethue.com/downloads/releases/4.0.1/hue-4.0.1.tgz2、安装依赖yum install ant asciidoc cyrus-sasl-devel cyrus-sasl-gssapi cyrus-sasl-plain gcc gcc-c++ krb5-devel libffi-devel libxml2-devel libxslt-devel make mysql-devel openldap-devel

一、安装1、下载#安装jdk,flume依赖于java环境wget http://mirrors.hust.edu.cn/apache/flume/1.6.0/apache-flume-1.6.0-bin.tar.gz2、解压tar zxf apache-flume-1.6.0-bin.tar.gz -C /opt/3、配置cd /opt/apache-flume-1.6.0-bin/confcp flume-conf.properties.template flume-conf.pro

一、安装前提1、HBase 依赖于 HDFS 做底层的数据存储2、HBase 依赖于 MapReduce 做数据计算3、HBase 依赖于 ZooKeeper 做服务协调4、HBase源码是java编写的，安装需要依赖JDK5、zookeeper和hdfs安装可以参考 hdfs分布式安装二、HBase的集群安装1、解压安装包#下载wget http://archive.apache.org/dist/hbase/1.2.6/hbase-1.2.6-bin.tar.gz#解压到指定目录

一、修改hive配置文件vim hive-site.xml<property> <name>hive.files.umask.value</name> <value>0002</value></property><property> <name>hive.metastore.authorization.storage.checks</name> <value>tr

1、登录mysql操作use hive;（1）修改表字段注解和表注解alter table COLUMNS_V2 modify column COMMENT varchar(256) character set utf8;alter table TABLE_PARAMS modify column PARAM_VALUE varchar(4000) character set utf8;（2）修改分区字段注解alter table PARTITION_PARAMS modify column

1、自定义认证需新添加一个jar#jar包我已经上传到我的资源里面了，有需要可以有偿下载cp hive-server2.jar /data/apache-hive-1.2.1-bin/lib2、修改hive-site.xml<property> <name>hive.server2.thrift.port</name> <value>10000</value></property><property&

报错Error: Could not open client transport with JDBC Uri: jdbc:hive2://node1:10000: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: root is not

一、安装数据库1、解压安装包tar zxf mysql-5.6.39-linux-glibc2.12-x86_64.tar.gz -C /data/2、修改文件目录名称mv /data/mysql-5.6.39-linux-glibc2.12-x86_64/ mysql3、创建数据目录mkdir sqldatauseradd -M -s /sbin/nologin mysql##授权cd /data/chown -R mysql.mysql mysql4、配置环境变量vim

服务节点分布node1NameNode、DFSZKFailoverController(ZKFC)、ResourceManager、JournalNode、ZooKeeper、QuorumPeerMainnode2NameNode、DFSZKFailoverController(ZKFC)、ResourceManager、JournalNode、ZooKeeper、QuorumPeerMainnode3DataNode、NodeManager、JournalNode、QuorumPeerMa

一，基础环境1、主机名配置vim /etc/hosts192.168.137.128 node12、安装JDKtar zxf /root/jdk1.8.0_111.tar.gz -C /opt/echo 'export PATH=$PATH:/opt/jdk1.8.0_101/bin' >>/etc/profilesource /etc/profile3、配置SSH无密码登陆ssh-keygenssh-copy-id -i id_rsa.pub root@node1

一，基础环境1、主机名配置hostnamectl set-hostname node1hostnamectl set-hostname node2hostnamectl set-hostname node3vim /etc/hosts192.168.137.128 node1192.168.137.129 node2192.168.137.130 node32、安装JDKtar zxf /root/jdk1.8.0_101.tar.gz -C /data/echo 'export