开始玩hadoop10 高可用(HA)配置;Hadoop2.6+HA+Zookeeper3.4.6+Hbase1.0.0

转载 2015年07月07日 17:43:11

应用地址在http://blog.csdn.net/onepiecehuiyu/article/details/45271493

这一块我觉得大大超出了我对hadoop 难度的预期;

额,我这里转了别人的一篇高可用的配置方法:

我觉得手头要有这么多台机器,也不是很容易啊(虚拟机的话怎么主机性能也要跟上才行啊 。。。只有以后有机会去公司里玩玩了)

core-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
	<property>
		<name>fs.defaultFS</name>
		<value>hdfs://cluster</value>
	</property>
        <property>
                <name>hadoop.tmp.dir</name>
                <value>/soft/hadoop-2.6.0/tmp/</value>
        </property>
	<property>
		<name>ha.zookeeper.quorum</name>
		<value>master1:2181,master2:2181,slave1:2181,slave2:2181,slave3:2181</value>
	</property>
	<property>
                <name>io.file.buffer.size</name>
                <value>131072</value>
        </property>
	<property>
                <name>dfs.ha.fencing.methods</name>
                <value>sshfence</value>
        </property>
        <property>
                <name>dfs.ha.fencing.ssh.private-key-files</name>
                <value>/root/.ssh/id_dsa</value>
        </property>
</configuration>



fairscheduler.xml

<?xml version="1.0"?>
<allocations>
         <queue name="news">
                 <minResources>1024 mb, 1 vcores </minResources>
                 <maxResources>1536 mb, 1 vcores </maxResources>
                 <maxRunningApps>5</maxRunningApps>
                 <minSharePreemptionTimeout>300</minSharePreemptionTimeout>
                 <weight>1.0</weight>
                 <aclSubmitApps>root,yarn,search,hdfs</aclSubmitApps>
         </queue>
         <queue name="crawler">
                 <minResources>1024 mb, 1 vcores</minResources>
                 <maxResources>1536 mb, 1 vcores</maxResources>
         </queue>
         <queue name="map">
                 <minResources>1024 mb, 1 vcores</minResources>
                 <maxResources>1536 mb, 1 vcores</maxResources>
         </queue>
</allocations>

hbase-env.sh


#
#/**
# * Copyright 2007 The Apache Software Foundation
# *
# * Licensed to the Apache Software Foundation (ASF) under one
# * or more contributor license agreements.  See the NOTICE file
# * distributed with this work for additional information
# * regarding copyright ownership.  The ASF licenses this file
# * to you under the Apache License, Version 2.0 (the
# * "License"); you may not use this file except in compliance
# * with the License.  You may obtain a copy of the License at
# *
# *     http://www.apache.org/licenses/LICENSE-2.0
# *
# * Unless required by applicable law or agreed to in writing, software
# * distributed under the License is distributed on an "AS IS" BASIS,
# * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# * See the License for the specific language governing permissions and
# * limitations under the License.
# */

# Set environment variables here.

# This script sets variables multiple times over the course of starting an hbase process,
# so try to keep things idempotent unless you want to take an even deeper look
# into the startup scripts (bin/hbase, etc.)

# The java implementation to use.  Java 1.7+ required.
# export JAVA_HOME=/usr/java/jdk1.6.0/

# Extra Java CLASSPATH elements.  Optional.
# export HBASE_CLASSPATH=

# The maximum amount of heap to use, in MB. Default is 1000.
# export HBASE_HEAPSIZE=1000

# Uncomment below if you intend to use off heap cache.
# export HBASE_OFFHEAPSIZE=1000

# For example, to allocate 8G of offheap, to 8G:
# export HBASE_OFFHEAPSIZE=8G

# Extra Java runtime options.
# Below are what we set by default.  May only work with SUN JVM.
# For more on why as well as other possible settings,
# see http://wiki.apache.org/hadoop/PerformanceTuning
export HBASE_OPTS="-XX:+UseConcMarkSweepGC"

# Uncomment one of the below three options to enable java garbage collection logging for the server-side processes.

# This enables basic gc logging to the .out file.
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps"

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH>"

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0_34+ and 1.7.0_2+.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export SERVER_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M"

# Uncomment one of the below three options to enable java garbage collection logging for the client processes.

# This enables basic gc logging to the .out file.
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps"

# This enables basic gc logging to its own file.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH>"

# This enables basic GC logging to its own file with automatic log rolling. Only applies to jdk 1.6.0_34+ and 1.7.0_2+.
# If FILE-PATH is not replaced, the log file(.gc) would still be generated in the HBASE_LOG_DIR .
# export CLIENT_GC_OPTS="-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<FILE-PATH> -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=1 -XX:GCLogFileSize=512M"

# See the package documentation for org.apache.hadoop.hbase.io.hfile for other configurations
# needed setting up off-heap block caching. 

# Uncomment and adjust to enable JMX exporting
# See jmxremote.password and jmxremote.access in $JRE_HOME/lib/management to configure remote password access.
# More details at: http://java.sun.com/javase/6/docs/technotes/guides/management/agent.html
# NOTE: HBase provides an alternative JMX implementation to fix the random ports issue, please see JMX
# section in HBase Reference Guide for instructions.

# export HBASE_JMX_BASE="-Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
# export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10101"
# export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10102"
# export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10103"
# export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10104"
# export HBASE_REST_OPTS="$HBASE_REST_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10105"

# File naming hosts on which HRegionServers will run.  $HBASE_HOME/conf/regionservers by default.
# export HBASE_REGIONSERVERS=${HBASE_HOME}/conf/regionservers

# Uncomment and adjust to keep all the Region Server pages mapped to be memory resident
#HBASE_REGIONSERVER_MLOCK=true
#HBASE_REGIONSERVER_UID="hbase"

# File naming hosts on which backup HMaster will run.  $HBASE_HOME/conf/backup-masters by default.
# export HBASE_BACKUP_MASTERS=${HBASE_HOME}/conf/backup-masters

# Extra ssh options.  Empty by default.
# export HBASE_SSH_OPTS="-o ConnectTimeout=1 -o SendEnv=HBASE_CONF_DIR"

# Where log files are stored.  $HBASE_HOME/logs by default.
# export HBASE_LOG_DIR=${HBASE_HOME}/logs

# Enable remote JDWP debugging of major HBase processes. Meant for Core Developers 
# export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8070"
# export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8071"
# export HBASE_THRIFT_OPTS="$HBASE_THRIFT_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8072"
# export HBASE_ZOOKEEPER_OPTS="$HBASE_ZOOKEEPER_OPTS -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=8073"

# A string representing this instance of hbase. $USER by default.
# export HBASE_IDENT_STRING=$USER

# The scheduling priority for daemon processes.  See 'man nice'.
# export HBASE_NICENESS=10

# The directory where pid files are stored. /tmp by default.
# export HBASE_PID_DIR=/var/hadoop/pids

# Seconds to sleep between slave commands.  Unset by default.  This
# can be useful in large clusters, where, e.g., slave rsyncs can
# otherwise arrive faster than the master can service them.
# export HBASE_SLAVE_SLEEP=0.1

# Tell HBase whether it should manage it's own instance of Zookeeper or not.
# export HBASE_MANAGES_ZK=true

# The default log rolling policy is RFA, where the log file is rolled as per the size defined for the 
# RFA appender. Please refer to the log4j.properties file to see more details on this appender.
# In case one needs to do log rolling on a date change, one should set the environment property
# HBASE_ROOT_LOGGER to "<DESIRED_LOG LEVEL>,DRFA".
# For example:
# HBASE_ROOT_LOGGER=INFO,DRFA
# The reason for changing default to RFA is to avoid the boundary case of filling out disk space as 
# DRFA doesn't put any cap on the log size. Please refer to HBase-5655 for more context.

export JAVA_HOME=/soft/jdk1.8.0_20
export HBASE_MANAGES_ZK=false
export HBASE_HEAPSIZE=8000
export HBASE_LOG_DIR=/soft/hbase-1.0.0/logs
export HBASE_HOME=/soft/hbase-1.0.0
export HADOOP_HOME=/soft/hadoop-2.6.0

hbase-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
/**
 *
 * Licensed to the Apache Software Foundation (ASF) under one
 * or more contributor license agreements.  See the NOTICE file
 * distributed with this work for additional information
 * regarding copyright ownership.  The ASF licenses this file
 * to you under the Apache License, Version 2.0 (the
 * "License"); you may not use this file except in compliance
 * with the License.  You may obtain a copy of the License at
 *
 *     http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
-->
<configuration>
<property>
	<name>hbase.rootdir</name>
	<value>hdfs://192.168.3.141:8020/hbase</value>
</property>
<property>
	<name>hbase.tmp.dir</name>
	<value>file:/soft/hbase-1.0.0/tmp</value>
</property>
<property>
	<name>hbase.cluster.distributed</name>
	<value>true</value>
</property>
<property>
	<name>hbase.zookeeper.quorum</name>
	<value>192.168.3.141,master2,slave1,slave2,slave3</value>
</property>
<property>
	<name>hbase.zookeeper.property.clientPort</name>
	<value>2181</value>
</property>
<property>
	<name>hbase.zookeeper.property.dataDir</name>
	<value>/soft/zookeeper-3.4.6/var/data</value>
</property>
<property>
	<name>hbase.master</name>
	<value>hdfs://slave1:60000</value>
</property>
</configuration>

hdfs-site.xml

<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
	<property>
                <name>dfs.nameservices</name>
                <value>cluster</value>
        </property>
        <property>
                <name>dfs.ha.namenodes.cluster</name>
                <value>master1,master2</value>
        </property>
        <property>
                <name>dfs.namenode.rpc-address.cluster.master1</name>
                <value>master1:8020</value>
        </property>
        <property>
                <name>dfs.namenode.rpc-address.cluster.master2</name>
                <value>master2:8020</value>
        </property>
        <property>
                <name>dfs.namenode.http-address.cluster.master1</name>
                <value>master1:50070</value>
        </property>
        <property>
                <name>dfs.namenode.http-address.cluster.master2</name>
                <value>master2:50070</value>
        </property>
	<property>
                 <name>dfs.namenode.servicerpc-address.cluster.master1</name>
                 <value>master1:53333</value>
         </property>
         <property>
                 <name>dfs.namenode.servicerpc-address.cluster.master2</name>
                 <value>master2:53333</value>
         </property>
        <property>
                <name>dfs.namenode.shared.edits.dir</name>
                <value>qjournal://master1:8485;master2:8485;slave1:8485;slave2:8485;slave3:8485/cluster</value>
        </property>
        <property>
                <name>dfs.client.failover.proxy.provider.cluster</name>
                <value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
        </property>
        <property>
                <name>dfs.journalnode.edits.dir</name>
                <value>/soft/hadoop-2.6.0/mydata/journal</value>
        </property>
	<property>
		<name>dfs.replication</name>
		<value>3</value>
	</property>
	<property>  
	        <name>dfs.namenode.name.dir</name>  
	        <value>file:/soft/hadoop-2.6.0/mydata/name</value>  
	</property>  
	<property>  
   		<name>dfs.datanode.data.dir</name>  
    		<value>file:/soft/hadoop-2.6.0/mydata/data</value>  
	</property>  
	<property>
	        <name>dfs.ha.automatic-failover.enabled</name>
		<value>true</value>
	</property>
	<property>  
		<name>dfs.webhdfs.enabled</name>  
		<value>true</value>
	</property>
	<property>  
		<name>dfs.journalnode.http-address</name>  
		<value>0.0.0.0:8480</value>  
	</property>  
	<property>  
		<name>dfs.journalnode.rpc-address</name>  
		<value>0.0.0.0:8485</value>  
	</property>
	<property>    
		<name>dfs.permissions</name>    
		<value>false</value>    
	</property>
</configuration>


mapred-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>
	<property>
		<name>mapreduce.framework.name</name>
		<value>yarn</value>
	</property>
	<property>
		<name>mapreduce.jobhistory.address</name>
		<value>master1:10020</value>
	</property>
	<property>
		<name>mapreduce.jobhistory.webapp.address</name>
		<value>master1:19888</value>
	</property>
</configuration>


regionservers

master2
master1
slave1
slave2
slave3


yarn-site.xml

<?xml version="1.0"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->
<configuration>
	<property>
		<name>yarn.resourcemanager.ha.rm-ids</name>
		<value>rm1,rm2</value>
	</property>
	<property>
		<name>yarn.resourcemanager.hostname.rm1</name>
		<value>master1</value>
	</property>
	<property>
		<name>yarn.resourcemanager.hostname.rm2</name>
		<value>master2</value>
	</property>
	<property>
		<name>yarn.resourcemanager.ha.id</name>
		<value>rm1</value>
	</property>
	<property>
                 <name>yarn.resourcemanager.address.rm1</name>
                 <value>${yarn.resourcemanager.hostname.rm1}:8032</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.scheduler.address.rm1</name>
                 <value>${yarn.resourcemanager.hostname.rm1}:8030</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.webapp.https.address.rm1</name>
                 <value>${yarn.resourcemanager.hostname.rm1}:8089</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.webapp.address.rm1</name>
                 <value>${yarn.resourcemanager.hostname.rm1}:8088</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.resource-tracker.address.rm1</name>
                 <value>${yarn.resourcemanager.hostname.rm1}:8025</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.admin.address.rm1</name>
                 <value>${yarn.resourcemanager.hostname.rm1}:8041</value>
         </property>

         <property>
                 <name>yarn.resourcemanager.address.rm2</name>
                 <value>${yarn.resourcemanager.hostname.rm2}:8032</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.scheduler.address.rm2</name>
                 <value>${yarn.resourcemanager.hostname.rm2}:8030</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.webapp.https.address.rm2</name>
                 <value>${yarn.resourcemanager.hostname.rm2}:8089</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.webapp.address.rm2</name>
                 <value>${yarn.resourcemanager.hostname.rm2}:8088</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.resource-tracker.address.rm2</name>
                 <value>${yarn.resourcemanager.hostname.rm2}:8025</value>
         </property>
         <property>
                 <name>yarn.resourcemanager.admin.address.rm2</name>
                 <value>${yarn.resourcemanager.hostname.rm2}:8041</value>
         </property>

	<property>
		<name>yarn.nodemanager.aux-services</name>
		<value>mapreduce_shuffle</value>
	</property>
	<property>
		<name>yarn.resourcemanager.ha.enabled</name>
		<value>true</value>
	</property> 
	<property>
		<name>yarn.resourcemanager.ha.automatic-failover.enabled</name>
		<value>true</value>
	</property>
	<property>
		<name>yarn.nodemanager.local-dirs</name>
	    	<value>/soft/hadoop-2.6.0/mydata/yarn/local</value>
	</property>
	<property>
                <name>yarn.nodemanager.log-dirs</name>
                <value>/soft/hadoop-2.6.0/mydata/yarn/log</value>
        </property>
	<property>
		<name>yarn.client.failover-proxy-provider</name>
		<value>org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider</value>
	</property> 
	<property> 
		<name>yarn.resourcemanager.zk-state-store.address</name> 
  		<value>master1:2181,master2:2181,slave1:2181,slave2:2181,slave3:2181</value>
	</property>
	<property>
		<name>yarn.resourcemanager.zk-address</name>
		<value>master1:2181,master2:2181,slave1:2181,slave2:2181,slave3:2181</value>
	</property>
	<property> 
	  	<name>yarn.resourcemanager.store.class</name> 
  		<value>org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore</value> 
	</property> 
	<property>
		<name>yarn.resourcemanager.cluster-id</name>
		<value>cluster</value>
	</property> 
	<property>
		<name>yarn.resourcemanager.recovery.enabled</name>
		<value>true</value>
	</property>
	<property>
		<name>yarn.resourcemanager.scheduler.class</name>
		<value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
	</property>
	<property>
		<name>yarn.scheduler.fair.allocation.file</name>
		<value>/soft/hadoop-2.6.0/etc/hadoop/fairscheduler.xml</value>
	</property>

</configuration>


zoo.cfg

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial 
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between 
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just 
# example sakes.
dataDir=/soft/zookeeper-3.4.6/var/data
dataLogDir=/soft/zookeeper-3.4.6/var/datalog
# the port at which the clients will connect
clientPort=2181
server.1=master1:2888:3888
server.2=master2:2888:3888
server.3=slave1:2888:3888
server.4=slave2:2888:3888
server.5=slave3:2888:3888
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the 
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1


参考网址是:http://blog.csdn.net/onepiecehuiyu/article/details/45271493





hadoop系列文档4-配置Yarn高可用HA

背景 之前有一篇高可用HDFS HA的配置文档,此文档类似上次,介绍如何配置高可用Yarn’s ResourceManager,在hadoop中默认只有一个ResourceManger,现在增加一个节...
  • u010237107
  • u010237107
  • 2016年03月12日 09:45
  • 2014

Hadoop 2.0 HA高可用集群配置详解

本文介绍了Hadoop2.0的HA高可用架构,并详细记录了HDFS HA和YARN HA集群的搭建过程,最后通过运行一个MR任务对集群功能进行测试。...
  • carl810224
  • carl810224
  • 2016年08月09日 10:30
  • 4689

Hadoop HA高可用集群搭建(2.7.2)

1.集群规划: 2.前期准备: 3.搭建zookeeper集群(drguo3/drguo4/drguo5)
  • Dr_Guo
  • Dr_Guo
  • 2016年03月24日 22:33
  • 12867

hive metastore ha 配置

1.需要在两台或多台机器上启动hive metastore 服务,这台机器上hive配置如下 javax.jdo.option.ConnectionURL jdbc:mysql...
  • zhoudetiankong
  • zhoudetiankong
  • 2016年09月18日 11:44
  • 1838

Hive HA使用说明及Hive使用HAProxy配置HA(高可用)

Hive HA使用说明及Hive使用HAProxy配置HA(高可用) http://www.aboutyun.com/thread-10938-1-1.html (出处: about云开发) ...
  • houzhizhen
  • houzhizhen
  • 2016年12月27日 13:55
  • 595

hadoop3.0高可用HA配置详解

hadoop3.0高可用HA配置详解
  • lxb1022
  • lxb1022
  • 2017年10月31日 15:00
  • 464

hadoop2.7.3 HA高可用集群安装

hadoop2.7.3 HA高可用集群安装 HDFS高可用性(NameNode) NN必须实现共享元数据才能保证无缝切换;共享元数据可以采用Linux提供的NFS服务,也可以使用Ha...
  • ProfoundOx
  • ProfoundOx
  • 2016年12月20日 22:16
  • 1552

CDH中配置HDFS HA、Hbase HA

最近又安装 hadoop 集群, 故尝试了一下配置 HDFS 的 HA,这里使用的是 QJM 的 HA 方案。 关于 hadoop 集群的安装部署过程你可以参考 使用yum安装CDH Hadoo...
  • furenjievip
  • furenjievip
  • 2015年03月01日 16:39
  • 1422

HiveServer2的高可用-HA配置

HiveServer2的高可用-HA配置   : http://lxw1234.com/archives/2016/05/675.htm
  • haohaixingyun
  • haohaixingyun
  • 2016年06月07日 08:22
  • 4182

Hadoop HDFS高可用(HA)

版本记录: 2016-07-19 凌晨 初稿 在Hadoop 1.x 中,Namenode是集群的单点故障,一旦Namenode出现故障,整个集群将不可用,重启或者开启一个新的Namenode才能够从...
  • bingduanlbd
  • bingduanlbd
  • 2016年07月19日 09:18
  • 1484
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:开始玩hadoop10 高可用(HA)配置;Hadoop2.6+HA+Zookeeper3.4.6+Hbase1.0.0
举报原因:
原因补充:

(最多只允许输入30个字)