基本安装步骤
安装包下载
从大数据组件下载地址下载以下组件安装包
- hadoop-2.6.0-cdh5.6.0.tar.gz
- hbase-1.0.0-cdh5.6.0.tar.gz
- zookeeper-3.4.5-cdh5.6.0.tar.gz
从java下载地址下载以下组件安装包 jdk-8u144-linux-x64.tar.gz
java环境
- 解压
tar zxvf jdk-8u144-linux-x64.tar.gz
- 配置环境变量
vim /etc/profile
export JAVA_HOME=/opt/cdh/jdk1.8.0_144 export JRE_HOME=$JAVA_HOME/jre export CLASSPATH=$JAVA_HOME/lib:$JRE_HOME/lib:$CLASSPATH export PATH=.:$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
source /etc/profile
SSH免密登陆
hadoop安装
- 解压
tar zxvf hadoop-2.6.0-cdh5.6.0.tar.gz
- 修改配置文件,位于目录hadoop-2.6.0-cdh5.6.0/etc/hadoop/下
- 修改core-site.xml
<configuration> <property> <name>hadoop.tmp.dir</name> <value>/opt/cdh/hadoop/tmp</value> </property> <property> <name>dfs.name.dir</name> <value>/opt/cdh/hadoop/name</value> </property> <property> <name>fs.default.name</name> <value>hdfs://ubuntu:9000</value> </property> </configuration>
- 修改mapred-site.xml
<configuration> <property> <name>mapred.job.tracker</name> <value>ubuntu:9001</value> </property> </configuration>
- 修改hdfs-site.xml
<configuration> <property> <name>dfs.data.dir</name> <value>/opt/cdh/hadoop/data</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.permissions</name> <value>false</value> </property> </configuration>
- 修改hadoop-env.sh
export JAVA_HOME=/opt/cdh/jdk1.8.0_144
- 修改core-site.xml
- 格式化namenode
hadoop-2.6.0-cdh5.6.0/bin/hadoop namenode -format
- 启动hdfs
hadoop-2.6.0-cdh5.6.0/sbin/start-all.sh
- 验证
jps
- 登录网址: http://ip:50070
zookeeper安装
- 解压
tar zxvf zookeeper-3.4.5-cdh5.6.0.tar.gz
- 修改配置文件,位于目录zookeeper-3.4.5-cdh5.6.0/conf/下
- zoo.cfg
tickTime=2000 initLimit=10 syncLimit=5 dataDir=/opt/cdh/zookeeper clientPort=2181
- zoo.cfg
- 启动zookeeper
zookeeper-3.4.5-cdh5.6.0/bin/zkServer.sh start
hbase安装
- 解压
tar zxvf hbase-1.0.0-cdh5.6.0.tar.gz
- 修改配置文件,位于目录hbase-1.0.0-cdh5.6.0/conf/下
- hbase-env.sh
export JAVA_HOME=/opt/cdh/jdk1.8.0_144 export HBASE_MANAGES_ZK=false
- regionservers
ubuntu
- hbase-site.xml
<configuration> <property> <name>hbase.cluster.distributed</name> <value>true</value> </property> <property> <name>hbase.rootdir</name> <value>hdfs://ubuntu:9000/hbase</value> </property> <property> <name>hbase.master</name> <value>hdfs://ubuntu:60000</value> </property> <property> <name>hbase.zookeeper.quorum</name> <value>ubuntu:2181</value> </property> <property> <name>hbase.zookeeper.property.dataDir</name> <value>/opt/cdh/zookeeper</value> </property> </configuration>
- hbase-env.sh
- 启动hbase
hbase-1.0.0-cdh5.6.0/bin/start-hbase.sh
- 验证
jps
hbase-1.0.0-cdh5.6.0/bin/hbase shell
- 登陆网址: http://ip:60010
IDEA连接hbase登陆
- 新建maven项目
- 修改pom.xml文件
<repositories>
<repository>
<id>cloudera</id>
<url>https://repository.cloudera.com/artifactory/cloudera-repos</url>
</repository>
</repositories>
<dependencies>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase</artifactId>
<version>1.0.0-cdh5.6.0</version>
</dependency>
<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>1.0.0-cdh5.6.0</version>
</dependency>
</dependencies>
- 按照百度搜索结果编写demo程序并执行
ERROR解决
- 先确认以下地址一致
linux下的/etc/hosts里的ip与windows下C:\Windows\System32\drivers\etc\hosts里的ip映射一致 - 禁用Hbase自带的zookeeper
- 修改hbase-env.sh里的
export HBASE_MANAGES_ZK=false
- 修改hbase-env.sh里的